[FIXED] Dropping infinite values from dataframes in pandas?

Issue

What is the quickest/simplest way to drop nan and inf/-inf values from a pandas.DataFrame without resetting mode.use_inf_as_null?

I'd like to be able to use the subset and how arguments of dropna, except with inf values considered missing, like:

df.dropna(subset=["col1", "col2"], how="all", with_inf=True)

Is this possible? Is there a way to tell dropna to include inf in its definition of missing values?

Solution

The simplest way would be to first replace() infs to NaN:

df.replace([np.inf, -np.inf], np.nan, inplace=True)

and then use the dropna():

df.replace([np.inf, -np.inf], np.nan, inplace=True) \
    .dropna(subset=["col1", "col2"], how="all")

For example:

In [11]: df = pd.DataFrame([1, 2, np.inf, -np.inf])

In [12]: df.replace([np.inf, -np.inf], np.nan, inplace=True)
Out[12]:
    0
0   1
1   2
2 NaN
3 NaN

The same method would work for a Series.

Answered By - Andy Hayden