Working With S3 Bucket Policies Using Python

S3 bucket policies help us grant different levels of access to the S3 bucket. Managing S3 bucket policies efficiently is necessary for achieving better security for data stored in S3 buckets. In this tutorial, we will learn how to manage s3 bucket policies using Python and boto3.

Setting up permissions for S3

For this tutorial to work, we will need an IAM user who has access to upload a file to S3. We can configure this user on our local machine using AWS CLI or use its credentials directly in Python script. We have already covered how to create an IAM user with S3 access. If you do not have this user setup, please follow that blog first and then continue with this blog.

List S3 Bucket Policies

First, we will learn how to list all bucket policies using Python. The code below will list all policies on the bucket, and if no policies are attached, it will raise an error.

def list_bucket_policies():
    """
    This function lists all policies attached to s3 bucket.
    :return: None
    """
    s3_client = boto3.client("s3")
    try:
        response = s3_client.get_bucket_policy(Bucket="testbucket-frompython-2")
        pprint(response)
    except ClientError as e:
        # if you do not have any policy attached to bucket it will throw error
        # An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicyStatus operation:
        # The bucket policy does not exist
        print("No policy attached to this bucket")

In the above code, we have not specified any user credentials. In such cases, boto3 uses your local machine’s default AWS CLI profile. You can also specify which profile should be used by boto3 if you have multiple profiles on your device. All you need to do is add the below line to your code.

# setting up default profile for session
boto3.setup_default_session(profile_name='PROFILE_NAME_FROM_YOUR_MACHINE')

Another option is to specify the access key ID and secret access key in the code. This is not a recommended approach, and I believe using IAM credentials directly in code should be avoided in most cases. If you have to do this, you can use the access key ID and secret access key in the code, as shown below.

s3 = boto3.client("s3",
                aws_access_key_id=ACCESS_KEY,
                aws_secret_access_key=SECRET_KEY)

Checking if S3 Bucket is Public or Private

Sometimes, we do not want to list all the policies on the bucket but check whether it has a public or private status. Boto3 has provided an easy way to do this.

def check_bucket_status():
    """
    This function checks if bucket has public access ot private access.
    :return: None
    """
    s3_client = boto3.client("s3")
    try:
        response = s3_client.get_bucket_policy_status(Bucket="testbucket-frompython-2")
        pprint(response["PolicyStatus"])
    except ClientError as e:
        # if you do not have any policy attached to bucket it will throw error
        # An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicyStatus operation:
        # The bucket policy does not exist
        print("No policy attached to this bucket")

The above code prints a directory with a single key, which shows the public status of the s3 bucket.

#sample output of above code
{'IsPublic': False}

Setting S3 bucket policy

Now that we know how to list s3 bucket policies, the next question is how to attach a new policy to our S3 bucket.

Let’s say we want to make all objects in that bucket public by default. This is useful when using S3 to serve static resources or host static websites.

First, we need a policy to make the S3 bucket public. All S3 policies are JSON documents. I have attached a sample policy below. Let me know if you need more details about creating policies in AWS.

def set_bucket_policy():
    """
    This function adds policy to bucket.
    :return: None
    """
    # policy for making all objects in bucket public by default
    public_policy = """{
      "Id": "Policy1577423306792",
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "Stmt1577423305093",
          "Action": "s3:*",
          "Effect": "Allow",
          "Resource": "arn:aws:s3:::testbucket-frompython-2/*",
          "Principal": {
            "AWS": [
              "*"
            ]
          }
        }
      ]
    }"""
    s3_client = boto3.client("s3")

    try:
        response = s3_client.put_bucket_policy(
            Bucket="testbucket-frompython-2", Policy=public_policy
        )
        pprint(response)
        # checking bucket status. This should show us s3 bucket is public now
        check_bucket_status()
    except ClientError as e:
        # if you do not have any policy attached to bucket it will throw error
        # An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicyStatus operation:
        # The bucket policy does not exist
        print(e)

When we run the above code, it will make ”test bucket-from python-2” public. We can also validate this on the S3 console. If we click on the permission tab for this bucket, we can check that the above policy is attached.

Removing all policies from the S3 bucket

In the last step, we will learn how to remove all policies from the S3 bucket. This is simple. All we have to do is call the delete_bucket_policy function.

def delete_bucket_policies():
    """
    This function deletes all policies attached to the bucket.
    :return: None
    """
    s3_client = boto3.client("s3")
    s3_client.delete_bucket_policy(Bucket="testbucket-frompython-2")
    # checking bucket policies. This should say no policies attached to this bucket.
    check_bucket_status()

Conclusion

In this tutorial, we have learned how easy it is to manage S3 bucket policies using Python. We have used the S3 client for this, but the same can also be achieved using the S3 resource class. You can find code for this blog in the GitHub repo. I hope you found this helpful. See you in the next bog.