[FIXED] extract data from google bucket

Issue

I just received a link with datasets contained in a google bucket (it seems like a xml). I would like to extract the data using jupyter notebook but I don't know how to do it. I've been reading a lot about it but I still can't read the data. Any ideas?

Thanks

Solution

On your local environment,

• Download gcloud SDK and install it
• Perform a gcloud init and then gcloud auth application-default login

Please, don't generate and download a service account key file, as you will be able to see in many tutorials, it's a bad security practice when you can use your user credentials

Then, in your notebook, you can use this piece of code

Import the library

%pip3 install google-cloud-storage

Then the code

from google.cloud import storage
client = storage.Client()
bucket = client.get_bucket('YOUR BUCKET NAME')
blob = bucket.get_blob('path/to/file.xxx')

# for text file
content = blod.download_as_string();
# for binary file
blod.download_to_filename('path/to/localfilename.xxx');
# You can use file writer (you can write in memory if you want, here in a filesystem)
with open('path/to/localfilename.xxx',w) as f:
  blod.download_to_file(f);

Does it work for you? If no, edit your question with your errors.

Answered By - guillaume blaquiere