Working With S3 Bucket Policies Using Python

S3 bucket policies help us grant different levels of access to the S3 bucket. Managing S3 bucket policies efficiently is necessary for achieving better security for data stored in S3 buckets. In this tutorial, we will learn how to manage s3 bucket policies using Python and boto3.

Setting up permissions for S3

For this tutorial to work, we will need an IAM user who has access to upload a file to S3. We can configure this user on our local machine using AWS CLI or we can use its credentials directly in python script. We have already covered this topic on how to create an IAM user with S3 access. If you do not have this user setup please follow that blog first and then continue with this blog.

List S3 Bucket Policies

First, we will learn how we can list down all bucket policies using python. The below code will list all policies on the bucket and if there are no policies attached it raises an error.

def list_bucket_policies():
    """
    This function lists all policies attached to s3 bucket.
    :return: None
    """
    s3_client = boto3.client("s3")
    try:
        response = s3_client.get_bucket_policy(Bucket="testbucket-frompython-2")
        pprint(response)
    except ClientError as e:
        # if you do not have any policy attached to bucket it will throw error
        # An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicyStatus operation:
        # The bucket policy does not exist
        print("No policy attached to this bucket")

In the above code, we have not specified any user credentials. In such cases, boto3 uses the default AWS CLI profile set up on your local machine. You can also specify which profile should be used by boto3 if you have multiple profiles on your machine. All you need to do is add the below line to your code.

# setting up default profile for session
boto3.setup_default_session(profile_name='PROFILE_NAME_FROM_YOUR_MACHINE')

Another option is you can specify the access key id and secret access key in the code itself. This is not recommended approach and I strongly believe using IAM credentials directly in code should be avoided in most cases. You can use access key id and secret access key in code as shown below, in case you have to do this.

s3 = boto3.client("s3", 
                aws_access_key_id=ACCESS_KEY,
                aws_secret_access_key=SECRET_KEY)

Checking if S3 Bucket is Public or Private

Sometimes, we do not want to list all the policies on the bucket but check if the bucket has a status of public or private. Boto3 has provided an easy way to check this.

def check_bucket_status():
    """
    This function checks if bucket has public access ot private access.
    :return: None
    """
    s3_client = boto3.client("s3")
    try:
        response = s3_client.get_bucket_policy_status(Bucket="testbucket-frompython-2")
        pprint(response["PolicyStatus"])
    except ClientError as e:
        # if you do not have any policy attached to bucket it will throw error
        # An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicyStatus operation:
        # The bucket policy does not exist
        print("No policy attached to this bucket")

The above code prints directory with a single key which shows the public status of the s3 bucket.

#sample output of above code
{'IsPublic': False}

Setting S3 bucket policy

Now that we know how to list s3 bucket policies, the next question is how can we attach a new policy to our S3 bucket?

Let us say we want to make all objects in that bucket public by default. This is useful when we are using S3 for serving static resources or hosting static websites with S3.

First, we will need a policy that will make the S3 bucket public. All policies in S3 are JSON documents. I have attached the sample policy below. Let me know if you need more details about how to create policies in AWS.

def set_bucket_policy():
    """
    This function adds policy to bucket.
    :return: None
    """
    # policy for making all objects in bucket public by default
    public_policy = """{
      "Id": "Policy1577423306792",
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "Stmt1577423305093",
          "Action": "s3:*",
          "Effect": "Allow",
          "Resource": "arn:aws:s3:::testbucket-frompython-2/*",
          "Principal": {
            "AWS": [
              "*"
            ]
          }
        }
      ]
    }"""
    s3_client = boto3.client("s3")

    try:
        response = s3_client.put_bucket_policy(
            Bucket="testbucket-frompython-2", Policy=public_policy
        )
        pprint(response)
        # checking bucket status. This should show us s3 bucket is public now
        check_bucket_status()
    except ClientError as e:
        # if you do not have any policy attached to bucket it will throw error
        # An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicyStatus operation:
        # The bucket policy does not exist
        print(e)

When we run the above code, it will make ” testbucket-frompython-2 ” public. We can also validate this on the S3 console. If we click on the permission tab for this bucket, we can check that the above policy is attached to the bucket.

Removing all policies from the S3 bucket

In the last step, we will learn how to remove all policies from the S3 bucket. This is simple. All we have to do is call the delete_bucket_policy function.

def delete_bucket_policies():
    """
    This function deletes all policies attached to the bucket.
    :return: None
    """
    s3_client = boto3.client("s3")
    s3_client.delete_bucket_policy(Bucket="testbucket-frompython-2")
    # checking bucket policies. This should say no policies attached to this bucket.
    check_bucket_status()

Conclusion

In this tutorial, we have learned how easy it is to manage S3 bucket policies using Python. We have used the S3 client for this but the same can be achieved using the S3 resource class as well. You can find code for this blog in the GitHub repo. I hope you found this useful. See you in the next bog.