[FIXED] Convert from RGB to LAB using pytorch? Opencv's color conversion does not match their own formula

Issue

In order to speed up some of my past opencv/numpy powered operations, I am attempting to translate them to PyTorch, to run on GPU. One of the first of these steps it RGB to LAB conversion.

I referred to:

• the opencv documentation on their own color conversion
• the Wikipedia page on CIE Lab, which offers one version of the algorithm (consistent with opencv's version when accounting for some refactoring and different XYZ starting ranges)
• this implementation from EdgeFool, which is already in PyTorch, but expects different value ranges and tensor shapes.

I have managed to produce this code:

import torch

def tensor_like(source_data, target_tensor):
    return torch.tensor(
        source_data,
        device=target_tensor.device,
        dtype=target_tensor.dtype,
    )


RGB_TO_XYZ_FACTOR_MATRIX = [
    [0.412453, 0.357580, 0.180423],
    [0.212671, 0.715160, 0.072169],
    [0.019334, 0.119193, 0.950227],
]


def cie_f(input_: torch.Tensor):
    e = 6 / 29
    return torch.where(
        input_ > e ** 3,
        input_ ** (1 / 3),
        input_ / (3 * e ** 2) + 4 / 29,
    )



def tensor_rgb_to_lab(input_: torch.Tensor):
    # input_ is expected to be RGB 0-255, and of shape B, C, H, W
    # Implemented based on formulas written here (may need to scroll down):
    # https://docs.opencv.org/4.x/de/d25/imgproc_color_conversions.html
    dtype = input_.dtype
    if not torch.is_floating_point(input_):
        input_ = input_.float()

    input_ = input_.permute(0, 2, 3, 1)
    rgb_colors = input_.reshape(-1, 3)
    rgb_colors /= 255.0

    # RGB -> XYZ
    factor_matrix = tensor_like(RGB_TO_XYZ_FACTOR_MATRIX, rgb_colors)
    xyz_colors = torch.mm(rgb_colors, factor_matrix.T)
    # xyz_colors.mul_(255)

    # XYZ -> LAB
    xyz_colors *= tensor_like([1 / 0.950456, 1.0, 1 / 1.088754], xyz_colors)
    f_xyz_colors = cie_f(xyz_colors)
    fxyz_to_lab_factor_matrix = tensor_like([
        [0.0, 500.0, 0.0],
        [116.0, -500.0, 200.0],
        [0.0, 0.0, -200.0],
    ], f_xyz_colors)
    lab_colors = torch.mm(f_xyz_colors, fxyz_to_lab_factor_matrix)
    lab_colors += tensor_like([-16, 128, 128], lab_colors)
    lab_colors[:, 0] *= 255 / 100

    lab_colors = lab_colors.view_as(input_).permute(0, 3, 1, 2)
    lab_colors = lab_colors.round_().clamp_(0, 255).type(dtype)
    return lab_colors

When comparing my output against cv2.cvtColor(image, cv2.COLOR_RGB2LAB), my code does produce the correct results with RGB colors like (000, 000, 000), (255, 255, 255), (000, 255, 255), (255, 000, 255), (255, 255, 000), (255, 000, 000), (000, 255, 000), (000, 000, 255), but it fails on many other cases (more than 99% of RGB colors, according to my test).

One such failing case is RGB(128, 128, 128), which open-cv says corresponds to Lab(137, 128, 128), but my code indicates Lab(194, 128, 128). This is an interesting failing case since only 1 value is wrong, and I can manually go through the different steps of the formula for L only in an attempt to isolate the problem:

g = np.array((0.412453+0.357580+0.180423, 0.212671+0.715160+0.072169,0.019334+0.119193+0.950227)) * 128/255
g[0]/=0.950456
g[2]/=1.088754
# g => array([0.50196078, 0.50196078, 0.50196078]). Makes sense since 128/255=0.50196078
l = 116*g[1]**(1/3)-16  # since 0.50196078>0.008856
# l => 76.18945600835188  This is already wrong according to online RGB converters
l *= 255/100
# l => 194.2831128212973  This contradicts opencv's value and correlates with mine

After quadruple-checking my code, I was starting to think opencv might have a bug/rounding error in their code, or that they were doing an extra step differently than they say on their documentation. So I have looked up some online RGB to Lab converters (1 and 2), and after re-adjusting the output L value to the 255 range, they agree with opencv. I guess this really points to a mistake on my part (unless the websites internally rely on opencv?).

Considering how the result comes out correct for RGB values of both 0 and 255 (or 0 and 1 when scaled to 0..1), I think that all of the ax+b operations must be correct, and the t**(1/3) must be the issue... could the industry be somehow using a different exponent, maybe due to performance benefit?

What am I missing here? I am using opencv 4.4.0.46

Solution

Instead of implementing this conversion yourself, you can use kornia's implementation: kornia.color.rgb_to_lab(image) seems to do exactly what you are trying to do in your tensor_rgb_to_lab function.

If you insist, you can checkout their implementation here.

Answered By - Shai