[FIXED] Random weights seem to get generated when trying to set weights in tensorflow

Issue

I'm trying to write code to manually set the weights in a keras network, but when I build the network, there seems to be additional weights.

>>> import numpy as np
>>> import tensorflow as tf
>>> from tensorflow import keras as ke
>>> from tensorflow.keras import layers
>>> lay=layers.Dense(1,activation="relu")
>>> lay.add_weight(shape=(1,),)
<tf.Variable 'Variable:0' shape=(1,) dtype=float32, numpy=array([-0.05657911], dtype=float32)>
>>> lay.set_weights(np.array([[0.5]]))
>>> lay.get_weights()
[array([0.5], dtype=float32)] # EXPEXCTD
>>> net=ke.Sequential([ke.Input(shape=(1,)),lay])
>>> net.get_weights()
[array([0.5], dtype=float32), array([[1.4100171]], dtype=float32), array([0.], dtype=float32)] # ACTUAL
>>> net.get_layer(index=0).get_weights()
[array([0.5], dtype=float32), array([[1.4100171]], dtype=float32), array([0.], dtype=float32)] # ACTUAL

As you can see, it created an extra 2 numpy arrays when I built the network. Why is this? What do these extra weights do? Are they the biases? Why would a network with only 2 neurons have 3 different weights? How should I set them?

Edit:

Someone suggested building the network first, then setting the weights. This also doesn't work:

>>> import numpy as np
>>> import tensorflow as tf
>>> from tensorflow import keras as ke
>>> from tensorflow.keras import layers
>>> key=layers.Dense(1,activation="sigmoid")
>>> net=ke.Sequential([ke.Input(shape=(1,)),key])
>>> key.set_weights(np.array([[0.5]]))
Traceback (most recent call last):
  File "<pyshell#44>", line 1, in <module>
    key.set_weights(np.array([[0.5]]))
  File "D:\Users\Student\AppData\Local\Programs\Python\Python38\lib\site-packages\keras\engine\base_layer.py", line 1832, in set_weights
    raise ValueError(
ValueError: You called `set_weights(weights)` on layer "dense" with a weight list of length 1, but the layer was expecting 2 weights. Provided weights: [[0.5]]...

It's very unclear what the shape of the array should be.

Solution

The solution is to set the weights after creating the network, but you have to set the weights without adding them first and they must be given as a list containing a 2 dimensional numpy array and a 1 dimensional numpy array. ie:

>>> import numpy as np
>>> import tensorflow as tf
>>> from tensorflow import keras as ke
>>> from tensorflow.keras import layers
>>> key=layers.Dense(1,activation="sigmoid")
>>> net=ke.Sequential([ke.Input(shape=(1,)),key])
>>> key.set_weights([np.array([[0.5]]), np.array([0.])])
>>> key.get_weights()
[array([[0.5]], dtype=float32), array([0.], dtype=float32)]

Answered By - Programmer S