[FIXED] Python Pandas apply function not being applied to every row when using variables from a DataFrame

Issue

I have this weird Pandas problem, when I use the apply function using values from a data frame, it only gets applied to the first row:

import pandas as pd

# main data frame - to be edited
headerData = [['dataA', 'dataB']]
valuesData = [[10, 20], [10, 20]]
dfData = pd.DataFrame(valuesData, columns = headerData)
dfData.to_csv('MainData.csv', index=False)

readMainDataCSV = pd.read_csv('MainData.csv')
print(readMainDataCSV)

#variable data frame - pull values from this to edit main data frame
headerVariables = [['varA', 'varB']]
valuesVariables = [[2, 10]]
dfVariables = pd.DataFrame(valuesVariables, columns = headerVariables)
dfVariables.to_csv('Variables.csv', index=False)

readVariablesCSV = pd.read_csv('Variables.csv')
readVarA = readVariablesCSV['varA']
readVarB = readVariablesCSV['varB']

def formula(x):
    return (x / readVarA) * readVarB

dfFormulaApplied = readMainDataCSV.apply(lambda x: formula(x))
print('\n', dfFormulaApplied)

Output:
    dataA  dataB
0   50.0  100.0
1    NaN    NaN

But when I just use regular variables (not being called from a data frame), it functions just fine:

import pandas as pd

# main data frame - to be edited
headerData = [['dataA', 'dataB']]
valuesData = [[10, 20], [20, 40]]
dfData = pd.DataFrame(valuesData, columns = headerData)
dfData.to_csv('MainData.csv', index=False)

readMainDataCSV = pd.read_csv('MainData.csv')
print(readMainDataCSV)

# variables
readVarA = 2
readVarB = 10

def formula(x):
    return (x / readVarA) * readVarB

dfFormulaApplied = readMainDataCSV.apply(lambda x: formula(x))
print('\n', dfFormulaApplied)

Output:
    dataA  dataB
0   50.0  100.0
1  100.0  200.0

Help please I'm pulling my hair out.

Solution

If you take readVarA and readVarB from the dataframe by selecting the column it is a pandas Series with an index, which gives a problem in the calculation (dividing a series by another series with a different index doesn't work). You can take the first value from the series to get the value like this:

def formula(x):
    return (x / readVarA[0]) * readVarB[0]

Answered By - pieterbons