30

Is it possible to recover a file I deleted from my S3 bucket?

21

According to the Amazon S3 Documentation:

Once deleted, there is no method to restore or undelete an object.

4
  • It's unfortunate S3 doesn't support snapshots. – churnd Mar 24 '10 at 11:40
  • 3
    To give hope to some people, nowadays you can undelete files if your bucket has versioning enabled. – jweyrich Jun 14 '17 at 15:37
  • how dammit....? – scytale Dec 4 '18 at 21:41
  • Unless you have versioning turned on then you can restore files. Amazon marks deleted files with a delete marker, and you just need to delete that marker and the file will become available again. aws.amazon.com/premiumsupport/knowledge-center/… – byustephen Apr 6 at 15:51
57

If you have versioning enable, yes you can! On a versioned bucket, a delete action on a file does not really delete it but it adds a version with a "Delete Marker". You can delete the delete marker with the AWS CLI:

aws s3api delete-object --bucket yourbucket-name --key "yourfile" --version-id id_of_the_delete_marker

You can get all the files in the bucket with

aws --output text s3api list-object-versions --bucket yourbucket-name > files.txt

If you want to undelete all the files in the bucket you can try:

echo '#!/bin/bash' > undeleteScript.sh && aws --output text s3api list-object-versions --bucket yourbucket-name | grep -E "^DELETEMARKERS" | awk '{FS = "[\t]+"; print "aws s3api delete-object --bucket yourbucket-name --key \42"$3"\42 --version-id "$5";"}' >> undeleteScript.sh && . undeleteScript.sh; rm -f undeleteScript.sh;
7
  • 1
    I appreciate this answer! Worked flawlessly. – tacotuesday Jan 4 '16 at 22:11
  • 1
    dude! your undelete all files script just saved my life. thank you. – herostwist Mar 16 '16 at 11:39
  • 1
    For the script above you need to add a --output=text if you want it to work. He's assuming you are outputting your AWS command via a .txt file and not .JSON file. – Brad Allison Aug 9 '17 at 19:51
  • 2
    Why is this not the accepted answer? @vonhogen please reconsider your choice :) – Janaka Bandara Sep 18 '18 at 4:05
  • 2
    If you want to restore files in only a specific set of "dirs" in S3 (keys in the bucket starting with a specific prefix, technically), add --prefix your/dir/path to the beginning of @Chris Cinelli's 3rd script line, right before the grep ... echo '#!/bin/bash' > undeleteScript.sh && aws --output text s3api list-object-versions --bucket yourbucket-name --prefix your/dir/path/ | grep -E "^DELETEMARKERS" | awk '{FS = "[\t]+"; print "aws s3api delete-object --bucket yourbucket-name --key \42"$3"\42 --version-id "$5";"}' >> undeleteScript.sh && . undeleteScript.sh; rm -f undeleteScript.sh; – NateH06 Apr 8 '20 at 5:44
3

Just updating this question as I was looking for the answer:

You can add VERSIONING to S3 buckets now. This will cause S3 to keep versions of an object even after deletion. Full documentation: http://docs.amazonwebservices.com/AmazonS3/latest/dev/Versioning.html

1
  • Is it possible to retrieve a deleted file over AWS Console? Can't find a way to do that. – Rafael Oliveira Jan 30 '14 at 22:25
1

You can now list object versions using the AWS console: http://docs.aws.amazon.com/AmazonS3/latest/UG/managing-objects-versioned-bucket.html

If you have the right permissions, then you could delete the delete marker (the latest version of a deleted object) and the result would be that the object is available again.

0

Just in case you would like to restore huge chunks of s3 objects in a more sustainable way, here is a python version and bash versions of the same solution that @chris Cinelli shared

Python Version, sourced from here

#!/usr/bin/env python
from datetime import datetime, timezone

import boto3

# ######################################
#
# Empty Bucket of all delete markers from all objects.
#
# ######################################

# -----------------------------------
# Enter these values here:
thebucket = "<bucket_name>"
access_key = "<access_key>"
secret_key = "<secret_key>"
region_name = "<region_name>" # us-east-1

# ------------------------------------

s3 = boto3.resource("s3", region_name=region_name, aws_access_key_id=access_key, aws_secret_access_key=secret_key)
s3client = boto3.client("s3", aws_access_key_id=access_key, aws_secret_access_key=secret_key)

# paginate 100000 at a time
page_size = 100000
folder_in_thebucket = "files/invoices"
paginator = s3client.get_paginator("list_object_versions")
pageresponse = paginator.paginate(
    Bucket=thebucket, Prefix=folder_in_thebucket, PaginationConfig={"MaxItems": page_size}
)
deleted_at = datetime(2019, 10, 22, 20, 0, 0, tzinfo=timezone.utc)


def restore_all(pages):
    # iter over the pages from the paginator
    for page in pages:
        # Find if there are any delmarkers
        if "DeleteMarkers" in page.keys():
            for each_delmarker in page["DeleteMarkers"]:
                if each_delmarker["IsLatest"] is True and each_delmarker["LastModified"] > deleted_at:
                    restore(each_delmarker)


def restore(delete_marker):
    # Create a resource for the version-object
    # and use .delete() to remove it.
    file_object_version = s3.ObjectVersion(thebucket, delete_marker["Key"], delete_marker["VersionId"])
    # I added this output just so I could watch the script run.
    print(f"Restoring {delete_marker}")
    # Lastly, lets remove the del marker and recover one of many files.
    file_object_version.delete()


if __name__ == "__main__":
    print(f"Restoring files deleted after {deleted_at} in {thebucket}/{folder_in_thebucket}.")
    restore_all(pageresponse)

Bash Version, sourced from here

#!/bin/bash

bucket=$1
prefix=$2

set -e

echo "Removing all latest delete markers in $bucket with prefix $prefix"

#versions=`aws s3api list-object-versions --bucket $bucket | jq '.Versions | .[] | select(.IsLatest | not)'`
latest_delete_markers=`aws s3api list-object-versions --bucket $bucket --prefix "$prefix" | jq '.DeleteMarkers | .[] | select(.IsLatest)'`

# echo "latest delete markers:"
# echo "$latest_delete_markers"

# echo "removing delete markers"
for marker in $(echo "${latest_delete_markers}" | jq -r '@base64'); do
    marker=$(echo ${marker} | base64 --decode)

    # echo "marker:"
    # echo "$marker"

    key=`echo $marker | jq -r .Key`
    versionId=`echo $marker | jq -r .VersionId `

    printf "Removing delete marker $key $versionId ... "

    aws s3api delete-object --bucket $bucket --key "$key" --version-id $versionId > /dev/null
    printf "✓\n"
    
#     echo $cmd
#     $cmd
done

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.