[FIXED] Pandas moving average where the window expands/contracts at the ends?

Issue

This has probably been answered somewhere but I am having a right brain fade on what you actually call this.

In Python/Pandas, if I use:

df['sginal'] = df['signal'].rolling(window=9).mean()

Then I get a sliding window, size 9, which averages to the central position in the window before moving to the next sample point. This obviously doesn't work well at the ends because there is not enough data points on one side of the window. So you get "nan" as an output and also when you compare to the original input signal, the output is shifted (lagging by 8).

In MATLAB I often used:

signal = smooth(signal, 9, 'moving');

The way this works is it will grow the window from the start and then shrink at the end. So until I get to 5th element in the data, the moving average window is not 9. Instead it started at 1, then 3, then 5, then 7 and then finally 9. At the end of the data, the window shrinks down again.

This way you do not get "nan" and you do not get any lagging output. I don't care that the initial and final handful of values essentially haven't been averaged at the full 9 samples. I don't know what you call this though.

Does Pandas have anything similar? should I look in NumPy or SciPy instead?

Solution

You could define min_periods to eliminate the nans.

df['signal'] = df['signal'].rolling(window=9, min_periods=1).mean()

Answered By - amance