Skip to content

Amazon AWS - S3

Nothing can be simpler to interact with Amazon Simple Storage Service (S3) than the BotCity plugin for AWS S3.

Interact with your buckets, files, folders and more using this simple and curated API.

Installation

pip install botcity-aws-s3-plugin

Importing the Plugin

After you installed this package, the next step is to import the package into your code and start using the functions.

from botcity.plugins.aws.s3 import BotAWSS3Plugin

Setting up connection

Caution

S3 does work only POSIX-style paths use "/" (forward slash). Relative paths are not allowed, if you are going to make any changes, put the absolute path of the file or folder.

Note

There are two different ways to authenticate.

1. Creating the .aws folder in the home directory, you need to create two files.

# ~/.aws/config
[default]
region=<region_code>

# ~/.aws/credentials
[default]
aws_access_key_id=<your_aws_access_key_id>
aws_secret_access_key=<your_aws_secret_access_key>

2. Passing credentials in the class constructor.

# Using the `.aws` folder
s3 = BotAWSS3Plugin()

# Alternative using the credentials as constructor arguments
s3 = BotAWSS3Plugin(
    use_credentials_file=False,
    access_key_id='<your_aws_access_key_id>',
    secret_access_key='<your_aws_secret_access_key>',
    region_name='<region_code>'
)

Bucket operations

Listing buckets

from botcity.plugins.aws.s3 import BotAWSS3Plugin, Filter

s3 = BotAWSS3Plugin()
print(s3.list_buckets())

print(s3.filter_buckets(text='test-', filter_=Filter.STARTS_WITH))

Creating bucket

Attention

Amazon S3 supports global buckets, which means that each bucket name must be unique across all AWS accounts in all the AWS Regions within a partition.

Tip

All methods using bucket_name parameter are optional and they can be defined like this. 

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()
s3.bucket_name = '<your_bucket_name>'

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()
s3.create_bucket(bucket_name='<your_bucket_name>')

# Or
# s3.bucket_name = '<your_bucket_name>'
# s3.create_bucket()

Deleting bucket

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()
s3.delete_bucket(bucket_name='<your_bucket_name>')

File operations

Uploading file

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()
s3.upload_file(
    bucket_name='<your_bucket_name>',
    file_path='/<path_to_file>/example.txt',
    bucket_filename='example.txt'
)

Downloading file

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()
filepath = s3.download_file(
    bucket_name='<your_bucket_name>',
    filename='example.txt',
    path_to_save='/<path_to_save>/'
)
print(filepath)  # Returns: /<path_to_save>/example.txt

Copying file

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()

# copying the file to another bucket
s3.copy_file(
    bucket_name='<your_bucket_name>',
    filename='example.txt',
    target_bucket_name='another_bucket',
    target_filename='copy_of_the_file_in_another_bucket.txt'
)

# copying the file to another place in the same bucket
s3.copy_file(
    bucket_name='<your_bucket_name>',
    filename='example.txt',
    # target_bucket_name='another_bucket', ignore this argument
    target_filename='copy_of_the_file_in_the_same_bucket.txt'
)

Listing files

from botcity.plugins.aws.s3 import BotAWSS3Plugin, Filter

s3 = BotAWSS3Plugin()
print(s3.list_files(bucket_name='<your_bucket_name>'))

print(s3.filter_files(bucket_name='<your_bucket_name>', text='.txt', filter_=Filter.ENDS_WITH))

Deleting file

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()
s3.delete_file(bucket_name='<your_bucket_name>', filename='example.txt')

Moving file

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()

# moving the file to another bucket
s3.move_file(
    bucket_name='<your_bucket_name>',
    filename='/test/current-file.txt',
    target_bucket_name='another_bucket',
    target_filename='/test/current-file.txt'
)

# copying the file to another place in the same bucket
s3.move_file(
    bucket_name='<your_bucket_name>',
    filename='/test/current-file.txt',
    # target_bucket_name='another_bucket', ignore this argument
    target_filename='/another_folder/current-file.txt'
)

Folder operations

Uploading folder

Attention

For now, empty folders are ignored.

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()
s3.upload_folder(
    bucket_name='<your_bucket_name>',
    bucket_folder='/docs',
    folder_path='/your_path_to/docs'
)

Downloading folder

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()
s3.download_folder(
    bucket_name='<your_bucket_name>',
    bucket_folder='/docs',
    path_to_save='/path_to_downloads/docs'
)

Copying folder

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()

# copying the folder to another bucket
s3.copy_folder(
    bucket_name='<your_bucket_name>',
    folder_name='/docs',
    target_bucket_name='another_bucket',
    target_folder_name='/docs'
)

# copying the folder to another place in the same bucket
s3.copy_folder(
    bucket_name='<your_bucket_name>',
    folder_name='/files/docs',
    # target_bucket_name='another_bucket', ignore this argument
    target_folder_name='/'  # will copy folder to bucket root path. /files/docs -> /docs
)

Deleting folder

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()
s3.delete_folder(
    bucket_name='<your_bucket_name>',
    folder_name='/docs'
)

Moving folder

Tip

The function to move folders can be used to rename a folder.

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()

# moving the folder to another bucket
s3.move_folder(
    bucket_name='<your_bucket_name>',
    folder_path='/docs/tests',
    target_bucket_name='another_bucket',
    target_folder_name='/'  # move /docs/tests to / in another bucket
)

# moving the folder to another place in the same bucket
s3.move_folder(
    bucket_name='<your_bucket_name>',
    folder_path='/docs',
    # target_bucket_name='another_bucket', ignore this argument
    target_folder_name='/copy-docs/'  # move folder to inside /copy-docs/
)

# renaming folder
s3.move_folder(
    bucket_name='<your_bucket_name>',
    folder_path='/docs',
    target_folder_name='/docs-v2'  # rename folder
)

Renaming folder

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()

# moving the folder to another bucket
s3.rename_folder(
    bucket_name='<your_bucket_name>',
    folder_path='/docs/tests',
    target_folder_name='tests-new-name'  # rename /docs/tests to /docs/tests-new-name
)

Other operations

To use other AWS S3 methods, use the S3 client property of the BotAWSS3Plugin class.

from botcity.plugins.aws.s3 import BotAWSS3Plugin

s3 = BotAWSS3Plugin()
result = s3.s3_client.get_bucket_acl(Bucket='<your_bucket_name>')
print(result)

Filter Enum

The filter operations (filter buckets, filter files) receive a Filter enum with possible values:

  • Filter.EQUALS
  • Filter.STARTS_WITH
  • Filter.ENDS_WITH
  • Filter.CONTAINS