[FIXED] Image intensity distribution changes during opencv warp affine

Issue

I am using python 3.8.5 and opencv 4.5.1 on windows 7

I am using the following code to rotate images.

def pad_rotate(image, ang, pad, pad_value=0):

    (h, w) = image.shape[:2]
    
    #create larger image and paste original image at the center.
    # this is done to avoid any cropping during rotation

    nH, nW = h + 2*pad, w + 2*pad  #new height and width

    cY, cX = nW//2, nH//2  #center of the new image

    #create new image with pad_values
    newImg = np.zeros((h+2*pad, w+2*pad), dtype=image.dtype)
    newImg[:,:] = pad_value

    #paste new image at the center
    newImg[pad:pad+h, pad:pad+w] = image

    #rotate CCW (for positive angles)
    M = cv2.getRotationMatrix2D(center=(cX, cY), angle=ang, scale=1.0)

    rotImg = cv2.warpAffine(newImg, M, (nW, nH), cv2.INTER_CUBIC, 
                        borderMode=cv2.BORDER_CONSTANT, borderValue=pad_value)

return rotImg

My issue is that after the rotation, image intensity distribution is different than original.

Following part of the question is edited to clarify the issue

img = np.random.rand(500,500)
Rimg = pad_rotate(img, 15, 300, np.nan)

Here is what these images look like:

Their intensities have clearly shifted:

np.percentile(img, [20, 50, 80])   
# prints  array([0.20061218, 0.50015415, 0.79989986])


np.nanpercentile(Rimg, [20, 50, 80])
# prints array([0.32420028, 0.50031483, 0.67656537])

Can someone please tell me how to avoid this normalization?

Solution

The averaging effect of the interpolation changes the distribution...

Note:

• There is a mistake in your code sample (not related to the percentiles).
  The 4'th argument of warpAffine is dst.
  replace cv2.warpAffine(newImg, M, (nW, nH), cv2.INTER_CUBIC with:

   cv2.warpAffine(newImg, M, (nW, nH), flags=cv2.INTER_CUBIC
  

I tried to simplify the code sample that reproduces the problem.
The code sample uses linear interpolation, 1 degree rotation, and no NaN values.

import numpy as np
import cv2

img = np.random.rand(1000, 1000)
M = cv2.getRotationMatrix2D((img.shape[1]//2, img.shape[0]//2), 1, 1)  # Rotate by 1 degree

Rimg = cv2.warpAffine(img, M, (img.shape[1], img.shape[0]), flags=cv2.INTER_LINEAR) # Use Linear interpolation

Rimg = Rimg[20:-20, 20:-20]  # Crop the part without the margins.

print(np.percentile(img, [20, 50, 80]))   #[0.20005696 0.49990526 0.79954818]
print(np.percentile(Rimg, [20, 50, 80]))  #[0.32244747 0.4998595  0.67698961]

cv2.imshow('img', img)
cv2.imshow('Rimg', Rimg)
cv2.waitKey()
cv2.destroyAllWindows()

---

When we disable the interpolation,
Rimg = cv2.warpAffine(img, M, (img.shape[1], img.shape[0]), flags=cv2.INTER_NEAREST)
The percentiles are: [0.19943713 0.50004768 0.7995525 ].

---

Simpler example for showing that averaging elements changes the distribution:

A = np.random.rand(10000000)
B = (A[0:-1:2] + A[1::2])/2  # Averaging every two elements.
print(np.percentile(A, [20, 50, 80]))  # [0.19995436 0.49999472 0.80007232]
print(np.percentile(B, [20, 50, 80]))  # [0.31617922 0.50000145 0.68377251]

---

Why does interpolation skews the distribution towered the median?

I am not a mathematician.
I am sure you can get a better explanation...

Here is an intuitive example:
Assume there is list of values with uniform distribution in range [0, 1].
Assume there is a zero value in the list:
[0.2, 0.7, 0, 0.5... ]

After averaging every two sequential elements, the probability for getting a zero element in the output list is very small (only two sequential zeros result a zero).

The example shows that averaging pushes the extreme values towered the center.

Answered By - Rotem