[FIXED] int8 data type in Pytorch

Issue

What is the best way to run a qauntized model using int8 data types in Pytorch? I know in pytorch I can define tensors as int8, however, when I actually want to use int8, I get:

RuntimeError: _thnn_conv2d_forward is not implemented for type torch.CharTensor

So I am confused, how to run quantized model in pytorch that uses for instance int8 when the datatype is not supported for computational blocks such as convolutions? I am using pytorch version 1.0.1.post2.

Solution

Depends what your goals are.

1. If you want to simulate your quantized model:

You may stick to existing float data type and only introduce truncation as needed, i.e.:

 x = torch.floor(x * 2**8) / 2**8

assuming x is a float tensor.

2. If you want to simulate your quantized model efficiently:

Then, I am afraid PyTorch will be not very useful since the low-level convolutional operator is implemented only for float type.

Answered By - penkovsky