[FIXED] Reproducibility tensorflow in python

Issue

I'm following this examples in order to tweak neuron networks in tensorflow. https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/01_neural_network_regression_in_tensorflow.ipynb

Specially for the model_1

# Set random seed
tf.random.set_seed(42)

# Replicate original model
model_1 = tf.keras.Sequential([
  tf.keras.layers.Dense(1)
])

# Compile the model
model_1.compile(loss=tf.keras.losses.mae,
                optimizer=tf.keras.optimizers.SGD(),
                metrics=['mae'])

# Fit the model
model_1.fit(tf.expand_dims(X_train, axis=-1), y_train, epochs=100)

So the model should be reproducible but : When I try with tf.random.set_seed(42) the model prediction change at each execution Whereas when I use tf.keras.utils.set_random_seed(42) the model is reproducible

Moreover I don't get the same result with this code

tf.keras.utils.set_random_seed(42)

# Create the model
model_1 = tf.keras.Sequential([
    tf.keras.layers.Dense(1)    
])

model_1.compile(
    loss=tf.keras.losses.mae,
    optimizer=tf.keras.optimizers.SGD(),
    metrics=["mae"]
)


model_1.fit(X_train,y_train,epochs=100)

example's one : mae = 30.638134 mse = 949.13086 mine : mae = 8.629547 mse = 80.97771

I checked that my training and testing data are the same. I checked that my model and the exemple's model are the same. Do you have some clues to figuring out ?

Training and testing sets :

X=tf.range(-100,100,delta=4)
y=X+10
X=tf.expand_dims(X,axis=1)
y=tf.expand_dims(y,axis=1)
X_train = X[:40]
y_train = y[:40]
X_test = X[40:]
y_test = y[40:]

Solution

According to the docs of tf.keras.utils.set_random_seed, it "sets all random seeds for the program", which includes random.seed, numpy.random.seed and tensorflow.random.seed. Going with your example, the following modified code creates deterministic results:

import tensorflow as tf
import random

X_train = tf.range(10)
y_train = tf.zeros(10)
# Set random seed
# tf.random.set_seed(42)
# tf.keras.utils.set_random_seed(42)
random.seed(42)
# Replicate original model
model_1 = tf.keras.Sequential([
  tf.keras.layers.Dense(1)
])

# Compile the model
model_1.compile(loss=tf.keras.losses.mae,
                optimizer=tf.keras.optimizers.SGD(),
                metrics=['mae'])
model_1.build(input_shape=(None, 1))
print(model_1.weights)

# Fit the model
model_1.fit(tf.expand_dims(X_train, axis=-1), y_train, epochs=100)

I included the model.build step to compare the model weight(s). They are the same every time.
It seems that tf.keras uses the random library to create the weights. Setting its seed to a fixed value creates the same results.

As for your question why this is different:

example's one : mae = 30.638134 mse = 949.13086 mine : mae = 8.629547 mse = 80.97771

You concluded rightly that tf.random.seed does not yield deterministic results. Therefore, the results it the notebook are random and not reproducible, as they set the wrong seed. If they would run the notebook again, they would get another result.

Answered By - mhenning