Issue
I have a DataFrame in pandas where some of the numbers are expressed in scientific notation (or exponent notation) like this:
id value
id 1.00 -4.22e-01
value -0.42 1.00e+00
percent -0.72 1.00e-01
played 0.03 -4.35e-02
money -0.22 3.37e-01
other NaN NaN
sy -0.03 2.19e-04
sz -0.33 3.83e-01
And the scientific notation makes what should be an easy comparison, needlessly difficult. I assume it's the 21900 value that's screwing it up for the others. I mean 1.0 is encoded. ONE!
This doesn't work:
np.set_printoptions(supress=True)
And pandas.set_printoptions
doesn't implement suppress either, and I've looked all at pd.describe_options()
in despair, and pd.core.format.set_eng_float_format()
only seems to turn it on for all the other float values, with no ability to turn it off.
Solution
Your data is probably object
dtype. This is a direct copy/paste of your data. read_csv
interprets it as the correct dtype. You should normally only have object
dtype on string-like fields.
In [5]: df = read_csv(StringIO(data),sep='\s+')
In [6]: df
Out[6]:
id value
id 1.00 -0.422000
value -0.42 1.000000
percent -0.72 0.100000
played 0.03 -0.043500
money -0.22 0.337000
other NaN NaN
sy -0.03 0.000219
sz -0.33 0.383000
check if your dtypes are object
In [7]: df.dtypes
Out[7]:
id float64
value float64
dtype: object
This converts this frame to object
dtype (notice the printing is funny now)
In [8]: df.astype(object)
Out[8]:
id value
id 1 -0.422
value -0.42 1
percent -0.72 0.1
played 0.03 -0.0435
money -0.22 0.337
other NaN NaN
sy -0.03 0.000219
sz -0.33 0.383
This is how to convert it back (astype(float)
) also works here
In [9]: df.astype(object).convert_objects()
Out[9]:
id value
id 1.00 -0.422000
value -0.42 1.000000
percent -0.72 0.100000
played 0.03 -0.043500
money -0.22 0.337000
other NaN NaN
sy -0.03 0.000219
sz -0.33 0.383000
This is what an object
dtype frame would look like
In [10]: df.astype(object).dtypes
Out[10]:
id object
value object
dtype: object
Answered By - Jeff
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.