Clash Royale CLAN TAG#URR8PPP

Get list of files using Boto from Private S3: 403 Forbidden

I would like to get a list of files that exist in a private S3. Generally, I do the following:

from boto.s3.connection import S3Connection
from boto.s3.key import Key

aws_access_key_id = "XXX"
aws_secret_access_key = "YYY"

conn = S3Connection(aws_access_key_id, aws_secret_access_key)
bucket = conn.get_bucket('bucket-name')

these_files = ["s3://bucket-name/" +
f.name for f in list(bucket.list("", ""))]


However, if I were to read the file in pandas I get 403 Forbidden Error. I found the solution here to get a file such as:

403 Forbidden Error

bucket = conn.get_bucket('bucket', validate=False)
k = bucket.get_key('file.csv')

df = pd.read_csv(k)


The problem with this solution is that it requires the user to know the file name in advance. I could avoid this by creating a custom function:

def readS3_files(f):
filename = f.split("/")[-1]
k = bucket.get_key(filename)
df = pd.read_csv(k)
return df


Is there something within Boto that will allow me to get the filenames from a private S3 bucket so I can simply read them in pandas without creating this custom function?

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.