Issue
I have the following code that creates a dask dataframe from an array. Problem is that all the types are converted to object. I tried to specify the metadata by couldn't find a way. How to specify meta in from_array?
b = np.array([(1.5, 2, 3, datetime(2000,1,1)), (4, 5, 6, datetime(2001, 2, 2))])
ddf = dd.from_array(b, columns=['col1', 'col2', 'col3', 'date1'], meta=['float', 'float', 'float', 'datetime'])
This throws AttributeError: 'list' object has no attribute '_constructor'
Solution
You could specify numpy array as a structured array:
import numpy as np
import pandas as pd
import dask.dataframe as dd
from datetime import datetime
b = np.array([(1.5, 2, 3, np.datetime64(datetime(2000,1,1))), (4, 5, 6, np.datetime64(datetime(2001, 2, 2)))], dtype=([('col1','float32'),('col2','float32'), ('col3','float32'), ('date1','<M8[us]') ]))
ddf = dd.from_array(b, columns=['col1', 'col2', 'col3', 'date1'])
ddf.head()
Answered By - Alexandra Dudkina
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.