Issue
I have a huge 3D (x,y,z) data set with (x,y) being the inputs and (z) is the output. Now the dataset is very large, and I need to use that information in real time with minimal delay.
Therefore, indexing/look-up table might seem slow. So my thought is to interpolate the dataset and in real time, instead of look-up table, I calcualte the value. So I don't have to store the original dataset but instead I can store the coefficients, which hopefully would be of smaller size than the original data set.
I used the scipy.interpolate.RectBivariateSpline
to perform interpolation. And I was able to fit the data and also obtain coefficients. But I am not sure how to reconstruct the interpolation function from the coefficients.
I want to emphesize that the interpolation function will only be evaluated at input (x,y). Generalization is not of concern here.
from scipy import interpolate
import numpy as np
x = np.arange(1,500)
y = np.arange(2,200)
X,Y = np.meshgrid(x,y)
z = np.sin(X+Y).T
a = interpolate.RectBivariateSpline(x,y,z)
# print(len(a.get_coeffs()))
# coefficients can be obtained by a.get_coeffs()
# I want to have the following
# f = construct_spline_from_coefficient(a.get_coeffs())
# z = f(x_old, y_old)
Another approach I had in mind is use deep neural network. Can anyone shed some light here? Is this an over-kill?
Solution
The solution is in the scipy official doc (link).
- Use
bisplrep
function (rep stands for representation) to obtain the interpoltaion outputtck
(see the docstring forbisplrep
). - The output
tck
is an array and can be stored in a file. - Use
bisplev
(ev stands for evaluation) to constrcut an function.
For using nueral network at interpolation see this state-of-the-art (paper) Training Neural Networks for and by Interpolation.
Answered By - Loading Zone
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.