Skip to content

How to Manage S3 Bucket Encryption Using Python

Posted on:August 2, 2022 at 05:17 AM

If we want to add a layer of security for our data in S3 buckets, we can encrypt that data. Instead of manually encrypting each file, S3 has provided a way to encrypt any data stored in the bucket by default using server-side encryption.

In this tutorial, we will learn how to manage s3 bucket encryption using Python and boto3 library.

Setting up permissions for S3

For this tutorial to work, we will need an IAM user who has access to upload a file to S3. We can configure this user on our local machine using AWS CLI or use its credentials directly in Python script. We have already covered how to create an IAM user with S3 access. If you do not have this user setup, please follow that blog first and then continue with this blog.

Setting Default Server Side Encryption for S3 Bucket

AWS S3 allows us to encrypt all data stored in S3 using AES-256 server-side encryption by default. We can enable this on a bucket, and any object uploaded to this bucket will be encrypted automatically.

def set_bucket_encryption():
    """
    This function sets up bucket level encryption.
    :return: None
    """
    s3_client = boto3.client("s3")
    response = s3_client.put_bucket_encryption(
        Bucket="testbucket-frompython-2",
        ServerSideEncryptionConfiguration={
            "Rules": [
                {"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}
            ]
        },
    )
    pprint(response)

The AES-256 encryption policy will be added to the S3 bucket when you run the above code. We can also validate this on the S3 console. For this, click on the bucket name, go to the properties tab, and scroll down until you find the encryption section.

s3-bucket-encryption-on-s3-console

When we set up server-side encryption on the S3 bucket, it only affects newly uploaded objects. Any unencrypted objects already in the S3 bucket will stay encrypted.

In the above code, we have not specified any user credentials. In such cases, boto3 uses your local machine’s default AWS CLI profile. You can also specify which profile should be used by boto3 if you have multiple profiles on your machine. All you need to do is add the below line to your code.

# setting up default profile for session
boto3.setup_default_session(profile_name='PROFILE_NAME_FROM_YOUR_MACHINE')

Another option is to specify the code’s access key ID and secret access key. Although this is not a recommended approach, and I strongly believe using IAM credentials directly in code should be avoided in most cases, if you have to do this, you can use the access key ID and secret access key in the code, as shown below.

s3 = boto3.client("s3",
                aws_access_key_id=ACCESS_KEY,
                aws_secret_access_key=SECRET_KEY)

Finding Current Bucket Encryption Policy using Python

In the last section, we learned how to set up encryption on a bucket level and checked the encryption attached to the S3 bucket on the S3 console. But what if we want to know the encryption status of the S3 bucket programmatically? There is an easy way with Python and boto3.

def get_bucket_encryption():
    """
    This function list encryption policy set up on s3 bucket.
    :return: None
    """
    s3_client = boto3.client("s3")
    # this function throws error if there is no encryption set up for this bucket.
    try:
        response = s3_client.get_bucket_encryption(Bucket="testbucket-frompython-2")
        pprint(response)
    except ClientError as e:
        print("No encryption present on this bucket.")

The above code should print the encryption status of S3 buckets. Remember, if bucket-level encryption is not set up, this function throws an error, which we must handle in our code.

Deleting S3 Bucket Encryption

Finally, we will learn how to delete S3 bucket encryption. This will remove default encryption from the S3 bucket. This change only affects new objects uploaded to that bucket. Any objects will stay encrypted even if we disable default bucket-level encryption.

def delete_bucket_encryption():
    """
    This function deletes encryption policy for this bucket.
    :return: None
    """
    s3_client = boto3.client("s3")
    response = s3_client.delete_bucket_encryption(Bucket="testbucket-frompython-2")
    pprint(response)
    get_bucket_encryption()

Conclusion

This tutorial taught us how to deal with S3 bucket encryption. AWS has made it very easy to encrypt our data when storing it in S3. I hope you have found this helpful. You can get code from this blog in this GitHub repo. See you in the next blog.