How to check any value Pandas DataFrame in is a NaN

How to check any value Pandas DataFrame in is a NaN

This article comes from  codeday  , author codeday

In Python  Pandas, check whether a DataFrame What is the best method (or more) NaN values are?

I know the function pd.isnan, but it returns a Boolean data frame for each element.  This POST here is not completely answer my question.

Best answer

jwilner response is the spot I was exploring whether there is a faster choice, because in my experience, find plane array (strangely) count faster than the code seems to be faster..:

 

df.isnull().values.any()
Copy the code

E.g:

 

In [2]: df = pd.DataFrame(np.random.randn(1000,1000))

In [3]: df[df > 0.9] = pd.np.nan

In [4]: %timeit df.isnull().any().any()
100 loops, best of 3: 14.7 ms per loop

In [5]: %timeit df.isnull().values.sum()
100 loops, best of 3: 2.15 ms per loop

In [6]: %timeit df.isnull().sum().sum()
100 loops, best of 3: 18 ms per loop

In [7]: %timeit df.isnull().values.any()
1000 loops, best of 3: 948 ?s per loop
Copy the code

df.isnull (). sum (). sum () a little slow, but of course there -NaN number of other information.