[FIXED] Concatenate 3d numpy arrays that have been chunked

Issue

I have a large number of 3d numpy arrays, which when assembled together, form a single contiguous 3d dataset*. However, the arrays were created by breaking the larger space into chunks. I need to assemble the chunk arrays back together. To simplify the problem, I've reduced it to the following example, with four chunks, each of which has 2x2x2 values.

So I have:

yellow_chunk = np.array([[[1,2], [5,6]], [[17,18], [21,22]]])
green_chunk = np.array([[[3,4], [7,8]], [[19,20], [23,24]]])
blue_chunk = np.array([[[9,10], [13,14]], [[25,26], [29,30]]])
red_chunk = np.array([[[11,12], [15,16]], [[27,28], [31,32]]])

And I want to end up with:

>>> output
array([[[ 1,  2,  3,  4],
        [ 5,  6,  7,  8],
        [ 9, 10, 11, 12],
        [13, 14, 15, 16]],

       [[17, 18, 19, 20],
        [21, 22, 23, 24],
        [25, 26, 27, 28],
        [29, 30, 31, 32]]])

Illustration for this small example:

Things I've tried

concatenate

>>> np.concatenate([yellow_chunk,green_chunk,blue_chunk,red_chunk],-1)
array([[[ 1,  2,  3,  4,  9, 10, 11, 12],
        [ 5,  6,  7,  8, 13, 14, 15, 16]],

       [[17, 18, 19, 20, 25, 26, 27, 28],
        [21, 22, 23, 24, 29, 30, 31, 32]]])

This was close, but the shape is wrong: 8x2x2 instead of the 4x2x4 I need.

hstack

>>> np.hstack([yellow_chunk,green_chunk,blue_chunk,red_chunk])
array([[[ 1,  2],
        [ 5,  6],
        [ 3,  4],
        [ 7,  8],
        [ 9, 10],
        [13, 14],
        [11, 12],
        [15, 16]],

       [[17, 18],
        [21, 22],
        [19, 20],
        [23, 24],
        [25, 26],
        [29, 30],
        [27, 28],
        [31, 32]]])

Also the wrong shape.

vstack

>>> np.vstack([yellow_chunk,green_chunk,blue_chunk,red_chunk])
array([[[ 1,  2],
        [ 5,  6]],

       [[17, 18],
        [21, 22]],

       [[ 3,  4],
        [ 7,  8]],

       [[19, 20],
        [23, 24]],

       [[ 9, 10],
        [13, 14]],

       [[25, 26],
        [29, 30]],

       [[11, 12],
        [15, 16]],

       [[27, 28],
        [31, 32]]])

Wrong shape and order.

dstack

>>> np.dstack([yellow_chunk,green_chunk,blue_chunk,red_chunk])
array([[[ 1,  2,  3,  4,  9, 10, 11, 12],
        [ 5,  6,  7,  8, 13, 14, 15, 16]],

       [[17, 18, 19, 20, 25, 26, 27, 28],
        [21, 22, 23, 24, 29, 30, 31, 32]]])

Wrong shape and order.

* In reality, I have 16x16 chunks, each of which has a shape of 16x128x16. So I'm stitching together "rows" of 256 values rather than the 4-value rows that I have in my small example above.

Solution

np.block([[yellow_chunk, green_chunk], [blue_chunk, red_chunk]])

>>>
[[[ 1  2  3  4]
  [ 5  6  7  8]
  [ 9 10 11 12]
  [13 14 15 16]]

 [[17 18 19 20]
  [21 22 23 24]
  [25 26 27 28]
  [29 30 31 32]]]

What you are doing here is assembling an nd-array from nested lists of blocks.

If you want more information about joining arrays, you can read this numpy.org doc on all the relevant methods and functions useable.

Answered By - DialFrost