dimanche 19 juin 2016

How to iterate through a S3 bucket using boto3? [duplicate]


This question already has an answer here:

I am trying to train a neural network where I pass in a series of images. I want to create a generator which passes each image in as a numpy array

from skimage import io
image_array = io.imread(url)

But this is only for a specific amazon aws url. I know the standard way using the boto library is something like this:

s3 = boto3.resource('s3')
s3.meta.client.download_file('mybucket', 'hello.txt', '/tmp/hello.txt') 

But again here, it seems like you are pointing towards a specific resource

I want something like this:

def my_generator():
    for object in s3_bucket():     # does an s3_bucket() iterator like this exist?
        image_array = io.imread(object)
        yield image_array

How could I do this?


Aucun commentaire:

Enregistrer un commentaire