[FIXED] How to hide index level in pandas <1.4

Issue

I'm trying to hide an index level from a pandas Styler with pandas 1.3.5. i.e. to replicate the behavior of hide(axis="index", level="level_name") in pandas 1.4.

Here is a minimale example of what I'm trying :

def multi_highlighter(row, range_colors):
    def find_color(value):
        for color, threshold in range_colors.items():
            if value < threshold:
                return color
        return "white"

    return [f"background-color: {find_color(v)}" for v in row]

range_colors = {"red": 18, "orange": 100}

data = pd.DataFrame({
    "Ex Date": ['2022-06-20', '2022-06-20', '2022-06-20', '2022-06-20', '2022-06-20', '2022-06-20', '2022-07-30', '2022-07-30', '2022-07-30'], 
    "Portfolio": ['UUU-SSS', 'UUU-SSS', 'UUU-SSS', 'RRR-DDD', 'RRR-DDD', 'RRR-DDD', 'KKK-VVV', 'KKK-VVV', 'KKK-VVV'],
    "Position": [120, 90, 110, 113, 111, 92, 104, 110, 110],
    "Strike": [18, 18, 19, 19, 20, 20, 15, 18, 19],
    })
(
    data
    .reset_index()
    .set_index(["Ex Date", "Portfolio", "index"])
    .style
    .apply(multi_highlighter, range_colors=range_colors, axis=1)
)

Adding some borders, it produces the following table:

Now to hide the "index" index level, I tried to achieve this by adding CSS styling from this answer, as follows:

.set_table_styles([{'selector': 'tr th:nth-child(3), tr td:nth-child(3)', 'props': [
    ('display', 'None')]}], overwrite=False)

But it gives me this results instead:

Solution

There are a few options in pandas 1.3.5, though this is a non-trivial operation without the use of the hide function available in 1.4.0.

Removing Index Levels >= 1

nth-child is not working here due to the use of rowspans in the MultiIndex. However, we can take advantage of the default CSS classes that are added by the Styler (style.py L158-L171):

• Index and Column names include index_name and level<k> where k is its level in a MultiIndex

• Index label cells include

  • row_heading
  • row<n> where n is the numeric position of the row
  • level<k> where k is the level in a MultiIndex

• Column label cells include

  • col_heading
  • col<n> where n is the numeric position of the column
  • level<k> where k is the level in a MultiIndex

• Blank cells include blank

So we can simply exclude the CSS selector .level2:not(.col_heading) where n is whatever level we want to hide (level0 would require a slight modification). We need to exclude the col_heading so we don't remove any column headers.

Additionally, we then need to remove one of the blank levels that so the top level lines up. I've taken the easy route and have chosen to remove the first blank for each row.

Note: This is not a durable solution and will be impacted by structure changes like MultiIndex columns.

Here is an example hiding level 2 with styles

hide_column_styles = [
    {
        # Remove all row values associated with level2
        'selector': f'th.level2:not(.col_heading)',
        'props': [('display', 'none')]
    },
    {
        # Remove the first th in each row if it is .blank
        'selector': 'thead th:first-child.blank',
        'props': [('display', 'none')]
    }
]

# Basic border
border_styles = [{
    'selector': '',
    'props': [('border-collapse', 'collapse')]
}, {
    'selector': 'table, th, td',
    'props': [('border', '1px solid black')]
}]
(
    data.reset_index()
        .set_index(["Ex Date", "Portfolio", "index"])
        .style
        .apply(multi_highlighter, range_colors=range_colors, axis=1)
        .set_table_styles([*hide_column_styles, *border_styles])
)

---

Removing Index level 0

If trying to remove level0 we only need to hide level0:

hide_column_styles = [
    {
        # Remove all values associated with level0 (including the first header row)
        'selector': f'th.level0:not(.col_heading)',
        'props': [('display', 'none')]
    }
]

The first index column will have the class .level0 and be hidden without needing an additional selector:

<tr>
  <th class="blank">&nbsp;</th>
  <th class="blank">&nbsp;</th>
  <th class="blank level0">&nbsp;</th> <!-- This will match and be hidden -->
  <th class="col_heading level0 col0">Position</th>
  <th class="col_heading level0 col1">Strike</th>
</tr>

---

Hiding levels that do not affect MultiIndex Uniqueness

If the index levels would remain unique after removing a level it is possible just to droplevel to remove the index before creating a Styler:

For example, here's an example of removing level 0 by dropping it.

n = 0  # level to drop
(
    data
        .reset_index()
        .set_index(["Ex Date", "Portfolio", "index"])
        .droplevel(level=n)  # Drop level from DataFrame
        .style
        .apply(multi_highlighter, range_colors=range_colors, axis=1)
        .set_table_styles([
        {
            'selector': 'table, th, td',
            'props': [('border', '1px solid black')]
        }
    ])
)

Note: this will only work if the MultiIndex is unique after the level is removed

If a level is dropped causing a non-unique MultiIndex (like level 2) a KeyError will occur when using Styler.apply or Styler.applymap:

KeyError: 'Styler.apply and .applymap are not compatible with non-unique index or columns.'

---

Pandas 1.4.0 and newer

Just to be super clear for any future readers of this question, this is a workaround for versions prior to 1.4.0. It is much simpler to use the hide function and the solution is much more durable than CSS selectors:

n = 2  # level to drop
border_styles = [{
    'selector': '',
    'props': [('border-collapse', 'collapse')]
}, {
    'selector': 'table, th, td',
    'props': [('border', '1px solid black')]
}]

(
    data
        .reset_index()
        .set_index(["Ex Date", "Portfolio", "index"])
        .style
        .apply(multi_highlighter, range_colors=range_colors, axis=1)
        .hide(axis=0, level=n)
        .set_table_styles(border_styles)
)

Answered By - Henry Ecker