remove a row from a dataframe if any row value is in another dataframe

aryan singh :

I have two dataframes

df1 with one column

{0:[1,2,3,4,5,6,7,11]}

df2 with two col

{0:[100,4,6,7],1:[1,3,4,7]}

i have to remove rows from df1 with value in any column in df2

ressult dataframe

df3 = [2,5,11]

kederrac :

you can use pandas.DataFrame.isin:

df1[~df1[0].isin(df2.values.flatten())]

output:

---

for large data frames, I have done the following simple benchmark:

import numpy as np

from simple_benchmark import BenchmarkBuilder
b = BenchmarkBuilder()

@b.add_function()
def anky_91(t):
    df1, df2 =  t
    df1[~df1[0].isin(df2.stack())]

@b.add_function()
def kederrac(t):  
    df1, df2 =  t
    df1[~df1[0].isin(df2.values.flatten())]

@b.add_function()   
def yatu(t):
    df1, df2 =  t
    df1[~df1.squeeze().isin(df2.stack())]


@b.add_arguments('NUmber of rows ind df')
def argument_provider():
    for exp in range(2, 18):
        size = 2**exp
        df1 = pd.DataFrame(np.random.randint(0, size // 10 or 10,size= (size , 1)))
        df2 = pd.DataFrame(np.random.randint(0, size // 10 or 10 ,size=(size , 2)))
        yield size, (df1, df2)

r = b.run()
r.plot()

Guess you like

Origin http://10.200.1.11:23101/article/api/json?id=401663&siteId=1

remove a row from a dataframe if any row value is in another dataframe

remove a row from a dataframe if any row value is in another dataframe , with dataframes having multiple columns

Pandas dataframe problem. Create column where a row cell gets the value of another row cell

Select a specific row based on the value of a column in the DataFrame

Find row number in column where it matches any other value in column of other dataframe

Create dataframe from row and colum indexes

Create dataframe from row and colum indexes

[Untitled] Row Operation DataFrame

pandas dataframe: is there any way to transform columns as row values in pandas

How to remove the first column of row number in Python pandas dataframe

pyspark dataframe get second lowest value for each row

Get values from between two other values for each row in the dataframe

成功解决A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,co

mapping values from one dataframe to another dataframe

Remove the largest value in each row

Dataframe cell to be locked and used for a running balance calculation conditional of result on another cell on same row

Pandas reads DataFrame data row by row and modifies corresponding data

Chinese name issue row aligned DataFrame

How to select row information in dataframe over ID

python add a row to pandas.DataFrame

[pandas] Python reads a row or column of DataFrame

python——dataframe get the specified row and column

DataFrame single row data keep format

How to select one row for each distinct value for a particular column and merge to form a new dataframe in Python?

How to select one row for each distinct value for a particular column and merge to form a new dataframe in Python?

Move data from row to another row within a group of specified rows

Concatenate A Row from columns of another table with multiple row

How to check any value Pandas DataFrame in is a NaN

Each row copy dataframe as the next line, and replicated multiple times

Efficient to covert a pandas dataframe to one hot based on data in each row