[FIXED] Your notebook tried to allocate more memory than is available. It has restarted

Issue

I was getting started with TalkingData AdTracking, my first entry to Kaggle competitions. The first line was pd.read_csv() and I got this error

Your notebook tried to allocate more memory than is available. It has restarted

I thought my code was run in the cloud and that I don't have to worry about the memory requirements. Can someone help me with that?

Solution

Yes, kaggle has a memory limit: Its 8GB per kernel, or 20 Min running. It takes a lot of server juice to host such a thing.

There are various solutions for this problem, notably loading and processing the dataset in chunks.

There is this as well. But I have no experience with it.

And of course you can use another cloud platform such as AWS, GCP, Oracle, etc..

Answered By - Sy Ker