How to Delete Files in S3 Bucket Using Python

In this series of blogs, we are learning how to manage S3 buckets and files using Python. In this tutorial, we will learn how to delete files in S3 bucket using python.

Setting up permissions for S3

For this tutorial to work, we will need an IAM user who has access to upload a file to S3. We can configure this user on our local machine using AWS CLI or we can use its credentials directly in python script. We have already covered this topic on how to create an IAM user with S3 access. If you do not have this user setup please follow that blog first and then continue with this blog.

Delete one file from the S3 bucket

First, we will learn how we can delete a single file from the S3 bucket. Below is code that deletes single from the S3 bucket.

def delete_object_from_bucket():
    bucket_name = "testbucket-frompython-2"
    file_name = "test9.txt"

    s3_client = boto3.client("s3")

    response = s3_client.delete_object(Bucket=bucket_name, Key=file_name)
    pprint(response)

Deleting multiple files from the S3 bucket

Sometimes we want to delete multiple files from the S3 bucket. Calling the above function multiple times is one option but boto3 has provided us with a better alternative. We can use the “delete_objects” function and pass a list of files to delete from the S3 bucket.

def delete_objects_from_bucket():
    bucket_name = "testbucket-frompython-2"

    s3_client = boto3.client("s3")
    response = s3_client.delete_objects(
        Bucket=bucket_name,
        Delete={"Objects": [{"Key": "text/test7.txt"}, {"Key": "text/test8.txt"}]},
    )
    pprint(response)

We can pass a list of Keys (file names) to this function and it will delete all those files in the single function call.

Delete all files in a folder in the S3 bucket

Now we want to delete all files from one folder in the S3 bucket. we can have 1000’s files in a single S3 folder. Both of the above approaches will work but these are not efficient and cumbersome to use when we want to delete 1000s of files.

Unfortunately, there is no simple function that can delete all files in a folder in S3. But we can create a workaround using the “delete_objetcs” function for this.

def delete_all_objects_from_s3_folder():
    """
    This function deletes all files in a folder from S3 bucket
    :return: None
    """
    bucket_name = "testbucket-frompython-2"

    s3_client = boto3.client("s3")

    # First we list all files in folder
    response = s3_client.list_objects_v2(Bucket=bucket_name, Prefix="images/")

    files_in_folder = response["Contents"]
    files_to_delete = []
    # We will create Key array to pass to delete_objects function
    for f in files_in_folder:
        files_to_delete.append({"Key": f["Key"]})

    # This will delete all files in a folder
    response = s3_client.delete_objects(
        Bucket=bucket_name, Delete={"Objects": files_to_delete}
    )
    pprint(response)

As shown in the above code, first we list down all files in a folder and use that list to delete all of those files.

Conclusion

In this tutorial, we have learned how to delete files from S3 bucket. I hope you have found this useful. You can code from this tutorial at the GitHub Repo. In the next blog, we will learn how to delete S3 bucket. See you there 🙂