[FIXED] Numpy reduce with char operations

Issue

I want a vectorized way to do:

numpy.char.add.reduce(["string_", values.astype("U"), "_string"], axis=0)

So, for example, I might want to send [0.1, 0.2, 0.3] to ["string_0.1_string", "string_0.2_string", "string_0.3_string"]. However, values will have an arbitrary shape and may be huge.

However, it doesn't seem like numpy.char.add is a ufunc, and one cannot just do numpy.char.add.reduce.

Is there a well-documented work around, or do I need to do something clever?

Solution

Your sample list/array:

In [71]: x=[0.1, 0.2, 0.3]; xx = np.array(x,'U5');xx
Out[71]: array(['0.1', '0.2', '0.3'], dtype='<U5')

As is clear from the docs, char.add is not a ufunc. It is just a shallow wrapper around the relevant string method.

But we can chain the add, same as if we did `'string_'+'0.01'+'_string'):

In [72]: np.char.add(np.char.add('string_',xx),'_string')
Out[72]: 
array(['string_0.1_string', 'string_0.2_string', 'string_0.3_string'],
      dtype='<U19')

numpy doesn't have special code for handling strings, so the char functions generally operate at the same speed as a list comprehension.

There is a char.join, but it operates on the characters of the array:

In [74]: np.char.join('_',xx)
Out[74]: array(['0_._1', '0_._2', '0_._3'], dtype='<U5')

Another approach is to use np.vectorize or np.frompyfunc with a utility function like:

In [75]: def foo(i):
    ...:     return '_'.join(['string',str(i),'string'])
    ...:     

In [76]: np.frompyfunc(foo,1,1)(x)
Out[76]: 
array(['string_0.1_string', 'string_0.2_string', 'string_0.3_string'],
      dtype=object)

For small arrays np.vectorize/frompyfunc tend to be a bit slower than list comprehensions, but they scale better for large arrays.

But the main advantage is that they handle multiple dimensions and broadcasting.

Answered By - hpaulj