How to Manage S3 Bucket Encryption Using Python

If we want to add an additional layer of security for our data in S3 buckets, we can encrypt that data. Instead of manually encrypting each file, S3 has provided a way to encrypt any data stored in the bucket by default using server-side encryption.

In this tutorial, we are going to learn how to manage s3 bucket encryption using python and boto3 library.

Setting up permissions for S3

For this tutorial to work, we will need an IAM user who has access to upload a file to S3. We can configure this user on our local machine using AWS CLI or we can use its credentials directly in python script. We have already covered this topic on how to create an IAM user with S3 access. If you do not have this user setup please follow that blog first and then continue with this blog.

Setting Default Server Side Encryption for S3 Bucket

AWS S3 provides us with an option to encrypt all data stored in S3 using AES-256 server-side encryption by default. We can enable this on a bucket and any object uploaded to this bucket will be encrypted automatically.

def set_bucket_encryption():
    """
    This function sets up bucket level encryption.
    :return: None
    """
    s3_client = boto3.client("s3")
    response = s3_client.put_bucket_encryption(
        Bucket="testbucket-frompython-2",
        ServerSideEncryptionConfiguration={
            "Rules": [
                {"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}
            ]
        },
    )
    pprint(response)

When you run the above code, this will add the AES-256 encryption policy to the S3 bucket. We can also validate this on the S3 console. For this click on bucket name and go to the properties tab and scroll down until you find the encryption section.

When we set up server-side encryption on the S3 bucket, it only affects new objects uploaded to that bucket. Any unencrypted objects already in the S3 bucket will stay encnrypted.

In the above code, we have not specified any user credentials. In such cases, boto3 uses the default AWS CLI profile set up on your local machine. You can also specify which profile should be used by boto3 if you have multiple profiles on your machine. All you need to do is add the below line to your code.

# setting up default profile for session
boto3.setup_default_session(profile_name='PROFILE_NAME_FROM_YOUR_MACHINE')

Another option is you can specify the access key id and secret access key in the code itself. This is not recommended approach and I strongly believe using IAM credentials directly in code should be avoided in most cases. You can use access key id and secret access key in code as shown below, in case you have to do this.

s3 = boto3.client("s3", 
                aws_access_key_id=ACCESS_KEY,
                aws_secret_access_key=SECRET_KEY)

Finding Current Bucket Encprytion Policy using Python

In the last section, we have learned how to set up encryption on a bucket level. We have also checked this encryption attached to the S3 bucket on the S3 console. But what if we want to programmatically know the encryption status of the S3 bucket? There is an easy way with python and boto3.

def get_bucket_encryption():
    """
    This function list encryption policy set up on s3 bucket.
    :return: None
    """
    s3_client = boto3.client("s3")
    # this function throws error if there is no encryption set up for this bucket.
    try:
        response = s3_client.get_bucket_encryption(Bucket="testbucket-frompython-2")
        pprint(response)
    except ClientError as e:
        print("No encryption present on this bucket.")

The above code should print S3 buckets encryption status. Remember, if there is no bucket-level encryption set up, this function throws an error and we have to handle it in our code.

Deleting S3 Bucket Encryption

Finally, we will learn how to delete S3 bucket encryption. This will remove default encryption from the S3 bucket. This change only affects new objects uploaded to that bucket. Any objects already encrypted will stay encrypted even if we disable default bucket level encprytion.

def delete_bucket_encryption():
    """
    This function deletes encryption policy for this bucket.
    :return: None
    """
    s3_client = boto3.client("s3")
    response = s3_client.delete_bucket_encryption(Bucket="testbucket-frompython-2")
    pprint(response)
    get_bucket_encryption()

Conclusion

In this tutorial, we have learned how to deal with S3 bucket encryption. AWS has made it very easy to encrypt our data when storing it to S3. I hope you have found this useful. You can get code from this blog in this GitHub repo. See you in the next blog.