Skip to content

How to Delete Files in S3 Bucket Using Python

Posted on:October 1, 2023 at 05:17 AM

In this series of blogs, we are learning how to manage S3 buckets and files using Python. In this tutorial, we will learn how to delete files in an S3 bucket using Python.

Setting up permissions for S3

For this tutorial to work, we will need an IAM user who has access to upload a file to S3. We can configure this user on our local machine using AWS CLI or use its credentials directly in Python script. We have already covered how to create an IAM user with S3 access. If you do not have this user setup, please follow that blog first and then continue with this blog.

Delete one file from the S3 bucket

First, we will learn how to delete a single file from the S3 bucket. Below is the code that deletes single from the S3 bucket.

def delete_object_from_bucket():
    bucket_name = "testbucket-frompython-2"
    file_name = "test9.txt"

    s3_client = boto3.client("s3")

    response = s3_client.delete_object(Bucket=bucket_name, Key=file_name)
    pprint(response)

Deleting multiple files from the S3 bucket

Sometimes, we want to delete multiple files from the S3 bucket. Calling the above function multiple times is one option, but boto3 has a better alternative. We can use the “delete_objects” function and pass a list of files to delete from the S3 bucket.

def delete_objects_from_bucket():
    bucket_name = "testbucket-frompython-2"

    s3_client = boto3.client("s3")
    response = s3_client.delete_objects(
        Bucket=bucket_name,
        Delete={"Objects": [{"Key": "text/test7.txt"}, {"Key": "text/test8.txt"}]},
    )
    pprint(response)

We can pass a list of Keys (file names) to this function, which will delete all those files in the single function call.

Delete all files in a folder in the S3 bucket

Now, we want to delete all files from one folder in the S3 bucket. We can have 1000 files in a single S3 folder. Both of the above approaches will work, but these are not efficient and cumbersome when we want to delete 1000s of files.

Unfortunately, there is no simple function that can delete all files in a folder in S3. But we can create a workaround using the “delete_objetcs” function.

def delete_all_objects_from_s3_folder():
    """
    This function deletes all files in a folder from S3 bucket
    :return: None
    """
    bucket_name = "testbucket-frompython-2"

    s3_client = boto3.client("s3")

    # First we list all files in folder
    response = s3_client.list_objects_v2(Bucket=bucket_name, Prefix="images/")

    files_in_folder = response["Contents"]
    files_to_delete = []
    # We will create Key array to pass to delete_objects function
    for f in files_in_folder:
        files_to_delete.append({"Key": f["Key"]})

    # This will delete all files in a folder
    response = s3_client.delete_objects(
        Bucket=bucket_name, Delete={"Objects": files_to_delete}
    )
    pprint(response)

As shown in the above code, we first list all files in a folder and then use that list to delete all of those files.

delete files in s3 bucket

Conclusion

This tutorial taught us how to delete files from the S3 bucket. I hope you have found this helpful. You can code from this tutorial at the GitHub Repo. In the next blog, we will learn how to delete S3 buckets. See you there 🙂