[FIXED] Concatenating Pandas Pivot Tables

Issue

I'm trying to convert a financial income statement spreadsheet into a table that I can easily plug into Power BI. The spreadsheet columns are as follows:

['Date', 'Budget Flag', 'Department', 'Account 1', 'Account 2', ..., 'Account N']

There are 78 columns of Accounts, and I need to consolidate. When I'm done, the spreadsheet columns need to be arranged like this:

['Date', 'Budget Flag', 'Department', 'Account Name', 'Value']

Here's what I've tried:

import pandas as pd
    
statement = pd.read_excel('Financial Dashboard Dataset.xlsx', sheet_name="Income Statement", header=0)

appended_data = []

for i in range(3,81):
    value = statement.columns[i]
    pivot = pd.pivot_table(statement, values=value, index=['Date', 'Budg', 'Depa'])
    pivot = pivot.assign(Account=value)
    appended_data.append(pivot)

appended_data = pd.concat(appended_data)

pivot.to_excel("final_product.xlsx", index=True, header=True)

So I'm essentially trying to pivot the values of each Account column against the three index columns, assign the Account column header as a value for each cell in a new column, and then union all of the pivot tables together.

My problem is that the resultant spreadsheet only contains the pivot table from the last account column. What gives?

Solution

The resultant spreadsheet only contains the pivot table from the last account column because that is the dataframe that I chose to write to excel.

I solved my problem by changing the last line to:

appended_data.to_excel("final_product.xlsx", index=True, header=True)

Sorry, all. It's been a long day. Hope this helps someone else!

Answered By - bannisterrb