[FIXED] pandas `to_numeric` integer downcast cast floats not to integer

Issue

With this sample dataframe:

>>> d = pd.DataFrame({'si': ['1', '2', 'NA'], 's': ['a', 'b', 'c']})

>>> d.dtypes
#
si    object
s     object
dtype: object

My first attempt was to use astype and the 'Int64' NA aware int type, but I got a

traceback

>>> d.si.astype('Int64')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-144-ed289e0c95aa> in <module>
----> 1 d.si.astype('Int64')
...

then I try the to_numeric method:

pandas to_numeric integer downcast cast floats

In [112]: d.loc[:, 'ii'] = pd.to_numeric(d.si, errors='coerce', downcast='integer')

In [113]: d.dtypes
Out[113]: 
si     object
s      object
ii    float64
dtype: object

In [114]: d
Out[114]: 
    si  s   ii
0    1  a  1.0
1    2  b  2.0
2   NA  c  NA

In the above I expect to have ii column with integers and integer nan

Documentation say:

downcast : {'integer', 'signed', 'unsigned', 'float'}, default None
    If not None, and if the data has been successfully cast to a
    numerical dtype (or if the data was numeric to begin with),
    downcast that resulting data to the smallest numerical dtype
    possible according to the following rules:

    - 'integer' or 'signed': smallest signed int dtype (min.: np.int8)
    - 'unsigned': smallest unsigned int dtype (min.: np.uint8)
    - 'float': smallest float dtype (min.: np.float32)

Solution

Unfortunately, pandas is still adapting/transitioning to fully supporting integer NaN. For that, you have to explicitly convert it to Int64 after your pd.to_numeric operation.

No need to downcast.

# Can also use `'Int64' as dtype below.
>>> pd.to_numeric(df['col'], errors='coerce').astype(pd.Int64Dtype())

# or

>>> pd.to_numeric(df['col'], errors='coerce').astype('Int64')

---

0       1
1       2
2       3
3    <NA>
Name: col, dtype: Int64

Answered By - rafaelc