[FIXED] Python - numpy.loadtxt how to ignore end commas?

Issue

I'm trying to read in a file that looks like this:

1, 2,
3, 4,

I'm using the following line:

l1,l2 = numpy.loadtxt('file.txt',unpack=True,delimiter=', ')

This gives me an error because the end comma in each row is lumped together as the last element (e.g. "2" is read as "2,"). Is there a way to ignore the last comma in each row, with loadtxt or another function?

Solution

numpy.genfromtxt is a bit more robust. If you use the default dtype (which is np.float64), it thinks there is a third column with missing values, so it creates a third column containing nan. If you give it dtype=None (which tells it to figure out the data type from the file), it returns a third column containing all zeros. Either way, you can ignore the last column by using usecols=[0, 1]:

In [14]: !cat trailing_comma.csv
1, 2,
3, 4,

Important note: I use delimiter=',', not delimiter=', '.

In [15]: np.genfromtxt('trailing_comma.csv', delimiter=',', dtype=None, usecols=[0,1])
Out[15]: 
array([[1, 2],
       [3, 4]])

In [16]: col1, col2 = np.genfromtxt('trailing_comma.csv', delimiter=',', dtype=None, usecols=[0,1], unpack=True)

In [17]: col1
Out[17]: array([1, 3])

In [18]: col2
Out[18]: array([2, 4])

Answered By - Warren Weckesser