[FIXED] Normalizing difference between x_train /= 255.0 and x_train = x_train/255.0

Issue

I have some simple code, which loads the mnist data and normalizes the images.

    mnist = tf.keras.datasets.mnist

    (x_train, y_train),(x_test, y_test) = mnist.load_data()
    x_train = x_train/255.0
    x_test = x_test/255.0

The code above works, however, if I try to use the shorthand for division, I get an error:

    mnist = tf.keras.datasets.mnist

    (x_train, y_train),(x_test, y_test) = mnist.load_data()
    x_train /= 255.0
    x_test /= 255.0

The error is as follows: TypeError: ufunc 'true_divide' output (typecode 'd') could not be coerced to provided output parameter (typecode 'B') according to the casting rule ''same_kind''

By playing around, I found a fix to it, in that typecasting x_train to float32, would get rid of the error, but I only stumbled upon the fix by accident. I don't understand why the code below fixes the issue

    mnist = tf.keras.datasets.mnist

    (x_train, y_train),(x_test, y_test) = mnist.load_data(path=path)
    x_train = x_train.astype('float32')
    x_test = x_test.astype('float32')
    x_train /= 255.0
    x_test /= 255.0

Could someone explain what's happening here? Why do the two versions behave differently? Why's an explicit case required in the second instance but not the first?
I didn't have much luck finding this behaviour documented anywhere.

Edit: I'm not sure what additional 'debugging details' I'm required to provide, since I've basically provided the entire code, the results as well as the details which I did not understand. I've also received no comments explaining why the question was closed, and/or what additional information is expected here. I would like some constructive criticism so as to atleast be able to ask the question in a better manner, if the present form isn't satisfactory by itself.

Solution

Augmented division (x_train /= 255.) isn't just shorthand to regular division. It is implemented differently than regular division and has different intent. Python objects have methods for implementing arithmetic functions.

• / is __truediv__(self, other): ==> object
• /= is __itruediv__(self, other): ==> object

Both return a result object (see Emulating numeric types in the python docs).

They are similar but have different intent when working with mutable objects like a list or a numpy array. __truediv__ should not modify self whereas __itruediv__ should. Implementations are free to ignore that, but generally c = a / b should not modify a but a /= b should modify a. When an object does not implement __itruediv__, python falls back to __truediv__ - immutable objects typically don't implement the augmented version because they can't modify themselves.

In the case of numpy arrays, augmented division (/=) ends up calling numpy.true_divide. Since augmented division modifies the original array, it uses the out parameter to broadcast the result into the original object. This is where the error is - you can't broadcast a different datatype into an exiting array. Augmented division is the same as

>>> import numpy as np
>>> x_train = np.array([1,2,3], dtype='B')
>>> np.true_divide(x_train, 255., out=x_train)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: ufunc 'true_divide' output (typecode 'd') could not be coerced to provided output parameter (typecode 'B') according to the casting rule ''same_kind''

where as standard division is the same as

>>> x_train = np.array([1,2,3], dtype='B')
>>> foo = np.true_divide(x_train, 255.)
>>> x_train = foo

In your case, the differences in the division isn't really a bug. Since your operation creates a different type, numpy needs to allocate a differently sized chunk of memory to hold it. You can either stick with non-augmented division or pre-convert the array like you did with float32. They are both reasonable approaches. The trick with the augmented operators is to note that they broadcast a result back to the original array and have all of the restrictions of the broadcast.

Answered By - tdelaney