[FIXED] How to compute sum over a tuple with unknown length in Cython

Issue

I want to understand why the following Cython code fails to compile. This function: dimsum simply computes sum of all dimensions of a numpy array:

from numpy cimport ndarray
import numpy as np

cpdef int dimsum(ndarray x):
    cdef:
        int N
        tuple shape
    
    shape = x.shape
    N = np.sum(shape)

    return N

The dimension of input array: x is unknown before run-time. As such, the shape of input array can not be unpacked and assigned to a few new variables.

I got the following error in the compilation:

Error compiling Cython file:
------------------------------------------------------------
...
cpdef int get_size(ndarray x):
    cdef:
        int N
        tuple shape

    shape = x.shape
            ^
------------------------------------------------------------

test_shape_tuple.pyx:9:13: Cannot convert 'npy_intp *' to Python object

Any suggestions on why this happens and how to resolve this?

Solution

Despite your mysterious intent with this function, you can solve this error by assigning the npy_intp* type to the shape variable directly.

A secondary performance-related issue with your code is the use of the np.sum function that forces a conversion to Python object.

A more efficient code would be (note that you could avoid defining the shape variable and reading x.shape directly in the for loop):

import numpy as np
cimport numpy as np
from numpy cimport ndarray, npy_intp


cpdef int dimsum(ndarray x):
    cdef:
        int i, out
        npy_intp* shape
    with nogil:
        shape = x.shape
        out = 0
        for i in range(x.ndim):
            out += shape[i]
    return out

Answered By - Saullo G. P. Castro