[FIXED] How to convert <class 'generator'> to numpy array

Issue

I want to shuffle the Pixel from an image in a random maner. With the following function everything is working like i want it but it is slow.

def randomize_image(img):
    # convert image from (m,n,3) to (N,3)
    rndImg = np.reshape(img, (img.shape[0]*img.shape[1], img.shape[2]))
    
    start_time = time.perf_counter()
    np.random.shuffle(rndImg)
    end_time = time.perf_counter()
    print('Time random shuffle: ', end_time - start_time)     
    
    rndImg = np.reshape(rndImg, img.shape)
    return rndImg

Thats why I changed it to the next function. It is also working the dtype is <class 'generator'> and no numpy array like in the first function. Is there a way to use the second function and to convert the result to an array?

thx

def rand_shuffle_faster(img):
    # convert image from (m,n,3) to (N,3)
    rndImg = np.reshape(img, (img.shape[0]*img.shape[1], img.shape[2]))
    length = rndImg.shape[1]
    yield rndImg[:, np.random.permutation(length)]
    rndImg = np.reshape(rndImg, img.shape)
    return rndImg

Solution

It sounds like you're ultimately after a faster version of your first function; I assume that turning the function into a Python generator with yield was not necessary and you only did that in search of a faster alternative. (Note that it appears faster only because it is not doing all the work.)

You are right that generating a permuted array of indices and using them to shuffle the image tends to be faster than using shuffle, though.

import numpy as np

def randomize_image(img):
    # convert image from (m,n,3) to (N,3)
    # Note "-1" means "whatever size is necessary"
    rndImg = np.reshape(img, (-1, img.shape[2]))
    np.random.shuffle(rndImg)    
    rndImg = np.reshape(rndImg, img.shape)
    return rndImg

def randomize_image2(img):
    # convert image from (m,n,3) to (N,3)
    rndImg = np.reshape(img, (-1, img.shape[2]))
    i = np.random.permutation(len(rndImg))
    rndImg = rndImg[i, :]
    rndImg = np.reshape(rndImg, img.shape)
    return rndImg

m, n = 1000, 1000
img = np.arange(m*n*3).reshape(m, n, 3)

img1 = randomize_image(img)
%timeit randomize_image1(img)
# 1.69 s ± 505 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

img2 = randomize_image2(img)
%timeit randomize_image2(img)
# 152 ms ± 31.2 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Depending on the size of your image, using a NumPy Generator to perform the permutation (instead of np.random.permutation) might be faster.

# include this outside of the function
rng = np.random.default_rng()
...
# inside the function, replace 
# i = np.random.permutation(len(rndImg)
# with:
i = rng.permutation(len(rndImg))

There are several other possibilities that could be faster in some situations, such as working along the last axis (which is contiguous in memory, assuming your image is a row-major array) and using np.argsort(rng.random(size=N)) to generate permutations, but they don't seem to be faster in general.

Answered By - Matt Haberland