33

Is it possible to recover a file I deleted from my S3 bucket?

5 Answers 5

23

According to the Amazon S3 Documentation:

Once deleted, there is no method to restore or undelete an object.

4
  • It's unfortunate S3 doesn't support snapshots.
    – churnd
    Mar 24, 2010 at 11:40
  • 4
    To give hope to some people, nowadays you can undelete files if your bucket has versioning enabled.
    – jweyrich
    Jun 14, 2017 at 15:37
  • how dammit....?
    – scytale
    Dec 4, 2018 at 21:41
  • 1
    Unless you have versioning turned on then you can restore files. Amazon marks deleted files with a delete marker, and you just need to delete that marker and the file will become available again. aws.amazon.com/premiumsupport/knowledge-center/…
    – byustephen
    Apr 6, 2021 at 15:51
62

If you have versioning enable, yes you can! On a versioned bucket, a delete action on a file does not really delete it but it adds a version with a "Delete Marker". You can delete the delete marker with the AWS CLI:

aws s3api delete-object --bucket yourbucket-name --key "yourfile" --version-id id_of_the_delete_marker

You can get all the files in the bucket with

aws --output text s3api list-object-versions --bucket yourbucket-name > files.txt

If you want to undelete all the files in the bucket you can try:

echo '#!/bin/bash' > undeleteScript.sh && aws --output text s3api list-object-versions --bucket yourbucket-name | grep -E "^DELETEMARKERS" | awk '{FS = "[\t]+"; print "aws s3api delete-object --bucket yourbucket-name --key \42"$3"\42 --version-id "$5";"}' >> undeleteScript.sh && . undeleteScript.sh; rm -f undeleteScript.sh;
7
  • 1
    I appreciate this answer! Worked flawlessly. Jan 4, 2016 at 22:11
  • 1
    dude! your undelete all files script just saved my life. thank you.
    – herostwist
    Mar 16, 2016 at 11:39
  • 1
    For the script above you need to add a --output=text if you want it to work. He's assuming you are outputting your AWS command via a .txt file and not .JSON file. Aug 9, 2017 at 19:51
  • 2
    Why is this not the accepted answer? @vonhogen please reconsider your choice :) Sep 18, 2018 at 4:05
  • 2
    If you want to restore files in only a specific set of "dirs" in S3 (keys in the bucket starting with a specific prefix, technically), add --prefix your/dir/path to the beginning of @Chris Cinelli's 3rd script line, right before the grep ... echo '#!/bin/bash' > undeleteScript.sh && aws --output text s3api list-object-versions --bucket yourbucket-name --prefix your/dir/path/ | grep -E "^DELETEMARKERS" | awk '{FS = "[\t]+"; print "aws s3api delete-object --bucket yourbucket-name --key \42"$3"\42 --version-id "$5";"}' >> undeleteScript.sh && . undeleteScript.sh; rm -f undeleteScript.sh;
    – NateH06
    Apr 8, 2020 at 5:44
3

Just updating this question as I was looking for the answer:

You can add VERSIONING to S3 buckets now. This will cause S3 to keep versions of an object even after deletion. Full documentation: http://docs.amazonwebservices.com/AmazonS3/latest/dev/Versioning.html

1
  • Is it possible to retrieve a deleted file over AWS Console? Can't find a way to do that. Jan 30, 2014 at 22:25
1

You can now list object versions using the AWS console: http://docs.aws.amazon.com/AmazonS3/latest/UG/managing-objects-versioned-bucket.html

If you have the right permissions, then you could delete the delete marker (the latest version of a deleted object) and the result would be that the object is available again.

1

Just in case you would like to restore huge chunks of s3 objects in a more sustainable way, here is a python version and bash versions of the same solution that @chris Cinelli shared

Python Version, sourced from here

#!/usr/bin/env python
from datetime import datetime, timezone

import boto3

# ######################################
#
# Empty Bucket of all delete markers from all objects.
#
# ######################################

# -----------------------------------
# Enter these values here:
thebucket = "<bucket_name>"
access_key = "<access_key>"
secret_key = "<secret_key>"
region_name = "<region_name>" # us-east-1

# ------------------------------------

s3 = boto3.resource("s3", region_name=region_name, aws_access_key_id=access_key, aws_secret_access_key=secret_key)
s3client = boto3.client("s3", aws_access_key_id=access_key, aws_secret_access_key=secret_key)

# paginate 100000 at a time
page_size = 100000
folder_in_thebucket = "files/invoices"
paginator = s3client.get_paginator("list_object_versions")
pageresponse = paginator.paginate(
    Bucket=thebucket, Prefix=folder_in_thebucket, PaginationConfig={"PageSize": page_size}
)
deleted_at = datetime(2019, 10, 22, 20, 0, 0, tzinfo=timezone.utc)


def restore_all(pages):
    # iter over the pages from the paginator
    for page in pages:
        # Find if there are any delmarkers
        if "DeleteMarkers" in page.keys():
            for each_delmarker in page["DeleteMarkers"]:
                if each_delmarker["IsLatest"] is True and each_delmarker["LastModified"] > deleted_at:
                    restore(each_delmarker)


def restore(delete_marker):
    # Create a resource for the version-object
    # and use .delete() to remove it.
    file_object_version = s3.ObjectVersion(thebucket, delete_marker["Key"], delete_marker["VersionId"])
    # I added this output just so I could watch the script run.
    print(f"Restoring {delete_marker}")
    # Lastly, lets remove the del marker and recover one of many files.
    file_object_version.delete()


if __name__ == "__main__":
    print(f"Restoring files deleted after {deleted_at} in {thebucket}/{folder_in_thebucket}.")
    restore_all(pageresponse)

Bash Version, sourced from here

#!/bin/bash

bucket=$1
prefix=$2

set -e

echo "Removing all latest delete markers in $bucket with prefix $prefix"

#versions=`aws s3api list-object-versions --bucket $bucket | jq '.Versions | .[] | select(.IsLatest | not)'`
latest_delete_markers=`aws s3api list-object-versions --bucket $bucket --prefix "$prefix" | jq '.DeleteMarkers | .[] | select(.IsLatest)'`

# echo "latest delete markers:"
# echo "$latest_delete_markers"

# echo "removing delete markers"
for marker in $(echo "${latest_delete_markers}" | jq -r '@base64'); do
    marker=$(echo ${marker} | base64 --decode)

    # echo "marker:"
    # echo "$marker"

    key=`echo $marker | jq -r .Key`
    versionId=`echo $marker | jq -r .VersionId `

    printf "Removing delete marker $key $versionId ... "

    aws s3api delete-object --bucket $bucket --key "$key" --version-id $versionId > /dev/null
    printf "✓\n"
    
#     echo $cmd
#     $cmd
done
1
  • Unfortunately there was no way to remove the prefix from Python script which would have been a great solution. To restore 90,000+ files it really does take 1/sec at a time.
    – Chirag
    Jan 11, 2022 at 8:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .