[FIXED] Human-readable or engineering style floats in Pandas?

Issue

How do I get human-readable floating point numbers output by Pandas? If I have numbers across several magnitudes of values, I would like to see the output printed in a concise format.

For example, with the code below, I have a table that has small fractional numbers, as well as large numbers with many zeroes. Setting precision to two decimals, the resulting output shows exponential numbers for the small and large magnitude numbers:

import numpy as np
import pandas as pd

np.random.seed(0)
pd.set_option('precision', 1)
columns = ['Small', 'Medium', 'Large']
df = pd.DataFrame(np.random.randn(4, 3), columns=columns)
df.Small = df.Small / 1000
df.Medium = df.Medium * 1000
df.Large = df.Large * 1000 * 1000 * 1000
print(df)

Output:

     Small  Medium    Large
0  1.8e-03   400.2  9.8e+08
1  2.2e-03  1867.6 -9.8e+08
2  9.5e-04  -151.4 -1.0e+08
3  4.1e-04   144.0  1.5e+09

Is there a way in Pandas to get this output more human-readable, like engineering format? I would expect output like the partial table below.

     Small  Medium    Large
0     1.8m   400.2   978.7M
...

Solution

Pandas has an engineering style floating point formatter that is not well-documented (only some documentation can be found about pd.set_eng_float_format()), based on matplotlib.ticker.EngFormatter.

This formatter can be used in two ways. The first way is to set the engineering format for all floats, the second way is to use the engineering formatter in the style object.

Set the engineering format for all floats:

np.random.seed(0)
pd.set_option('precision', 1)
columns = ['Small', 'Medium', 'Large']
df = pd.DataFrame(np.random.randn(4, 3), columns=columns)
df.Small = df.Small / 1000
df.Medium = df.Medium * 1000
df.Large = df.Large * 1000 * 1000 * 1000
pd.set_eng_float_format(accuracy=1, use_eng_prefix=True)
print(df)

Output:

    Small  Medium   Large
0    1.8m   400.2  978.7M
1    2.2m    1.9k -977.3M
2  950.1u  -151.4 -103.2M
3  410.6u   144.0    1.5G

The underlying formatter can also be used in style objects, either for all columns or with a formatter dictionary. Note that in the latter example, the Small column gets reduced to 0.0:

eng_fmt = pd.io.formats.format.EngFormatter(accuracy=1, use_eng_prefix=True)
style_all = df.style.format(formatter=eng_fmt)
style_large = df.style.format(formatter={'Large': eng_fmt})
# style_large.to_html()

Answered By - VirtualScooter