Issue
I am webscraping some data from a few websites, and using pandas to modify it.
On the first few chunks of data it worked well, but later I get this error message:
Traceback(most recent call last):
File "data.py", line 394 in <module> df2[['STATUS_ID_1','STATUS_ID_2']] = df2['STATUS'].str.split(n=1, expand=True)
File "/home/web/.local/lib/python2.7/site-packages/pandas/core/frame.py, line 2326, in __setitem__ self._setitem_array(key,value)
File "/home/web/.local/lib/python2.7/site-packages/pandas/core/frame.py, line 2350, in _setitem_array
raise ValueError("Columns must be same length as key') ValueError: Columns must be same length as key
My code is here:
df2 = pd.DataFrame(datatable,columns = cols)
df2['FLIGHT_ID_1'] = df2['FLIGHT'].str[:3]
df2['FLIGHT_ID_2'] = df2['FLIGHT'].str[3:].str.zfill(4)
df2[['STATUS_ID_1','STATUS_ID_2']] = df2['STATUS'].str.split(n=1, expand=True)
EDIT-jezrael : i used your code, and maked a print from this: I hope with this we can find where is the problem..because it seems it is randomly when the scripts has got a problem with this split..
0 1
2 Landed 8:33 AM
3 Landed 9:37 AM
4 Landed 9:10 AM
5 Landed 9:57 AM
6 Landed 9:36 AM
8 Landed 8:51 AM
9 Landed 9:18 AM
11 Landed 8:53 AM
12 Landed 7:59 AM
13 Landed 7:52 AM
14 Landed 8:56 AM
15 Landed 8:09 AM
18 Landed 8:42 AM
19 Landed 9:39 AM
20 Landed 9:45 AM
21 Landed 7:44 AM
23 Landed 8:36 AM
27 Landed 9:53 AM
29 Landed 9:26 AM
30 Landed 8:23 AM
35 Landed 9:59 AM
36 Landed 8:38 AM
37 Landed 9:38 AM
38 Landed 9:37 AM
40 Landed 9:27 AM
43 Landed 9:14 AM
44 Landed 9:22 AM
45 Landed 8:18 AM
46 Landed 10:01 AM
47 Landed 10:21 AM
.. ... ...
316 Delayed 5:00 PM
317 Delayed 4:34 PM
319 Estimated 2:58 PM
320 Estimated 3:02 PM
321 Delayed 4:47 PM
323 Estimated 3:08 PM
325 Delayed 3:52 PM
326 Estimated 3:09 PM
327 Estimated 2:37 PM
328 Estimated 3:17 PM
329 Estimated 3:20 PM
330 Estimated 2:39 PM
331 Delayed 4:04 PM
332 Delayed 4:36 PM
337 Estimated 3:47 PM
339 Estimated 3:37 PM
341 Delayed 4:32 PM
345 Estimated 3:34 PM
349 Estimated 3:24 PM
356 Delayed 4:56 PM
358 Estimated 3:45 PM
367 Estimated 4:09 PM
370 Estimated 4:04 PM
371 Estimated 4:11 PM
373 Delayed 5:21 PM
382 Estimated 3:56 PM
384 Delayed 4:28 PM
389 Delayed 4:41 PM
393 Estimated 4:02 PM
397 Delayed 5:23 PM
[240 rows x 2 columns]
Solution
You need a bit modify solution, because sometimes it return 2 and sometimes only one column:
df2 = pd.DataFrame({'STATUS':['Estimated 3:17 PM','Delayed 3:00 PM']})
df3 = df2['STATUS'].str.split(n=1, expand=True)
df3.columns = ['STATUS_ID{}'.format(x+1) for x in df3.columns]
print (df3)
STATUS_ID1 STATUS_ID2
0 Estimated 3:17 PM
1 Delayed 3:00 PM
df2 = df2.join(df3)
print (df2)
STATUS STATUS_ID1 STATUS_ID2
0 Estimated 3:17 PM Estimated 3:17 PM
1 Delayed 3:00 PM Delayed 3:00 PM
Another possible data - all data have no whitespaces and solution working too:
df2 = pd.DataFrame({'STATUS':['Canceled','Canceled']})
and solution return:
print (df2)
STATUS STATUS_ID1
0 Canceled Canceled
1 Canceled Canceled
All together:
df3 = df2['STATUS'].str.split(n=1, expand=True)
df3.columns = ['STATUS_ID{}'.format(x+1) for x in df3.columns]
df2 = df2.join(df3)
Answered By - jezrael
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.