[FIXED] model predicts NaN

Issue

I am trying to learn and practice on Tensorflow.js. So, I tried to train a neural network on a [,2] shaped array as x (as I understood, this would simulate a problem where I have x samples that each one has 2 variables) and a [,1] array as y (what would mean if I'm correct, that the combination of my 2 variables generate 1 output).

And I tried to code it:

const model = tf.sequential();
        model.add(tf.layers.dense({ units: 2, inputShape: [2] }));
        model.add(tf.layers.dense({ units: 64, inputShape: [2] }));
        model.add(tf.layers.dense({ units: 1, inputShape: [64] }));
        // Prepare the model for training: Specify the loss and the optimizer.
        model.compile({ loss: 'meanSquaredError', optimizer: 'sgd' });

        // Generate some synthetic data for training.
        const xs = tf.tensor([[1,5], [2,10], [3,15], [4,20], [5,25], [6,30], [7,35], [8,40]], [8, 2]);
        const ys = tf.tensor([1, 2, 3, 4, 5, 6, 7, 8], [8, 1]);

        // Train the model using the data.
        model.fit(xs, ys, { epochs: 100 }).then(() => {
            // Use the model to do inference on a data point the model hasn't seen before:
            // Open the browser devtools to see the output
            model.predict(tf.tensor([10, 50], [1, 2])).print();
        });

But, what I am facing is that when I try to predict the [10,50] input, I have the following console output:

Tensor [[NaN],]

So, I think my problem might be very simple, but I am really stuck with this and probably it is a matter of some background knowledge I'm missing.

Thank you!

Solution

The first layer takes the shape of the input data

model.add(tf.layers.dense({ units: 2, inputShape: [2] }))

The inputShape is [2], which means that your input x is of shape [2]. The last layer unit value gives the dimension of the output y.

 model.add(tf.layers.dense({ units: 1, inputShape: [64] }));

So the shape of y should be [1]

In this case, the NaN prediction is related to the number of epochs for your training. If you decrease it to 2 or 3, it will return a numerical value. Actually, the error is related to how your optimizer is updating the weights. Alternatively, you can change the optimizer to adam and it will be fine.

Answered By - edkeveked