0

How to reproduce:

  1. At hour 0, take a snapshot of a 100GB disk.
  2. At hour 12:
    1. Create a new EC2 instance.
    2. Create a volume from the snapshot.
    3. Attach the new volume to the new instance.
    4. Leave the instance running.

At this point I would expect all the data to be copied from the snapshot to the new volume in less than an hour, but it seems to take more in the order of 12 hours. In the meantime the new instance is unusable for production traffic, since any request may take a very long time to serve.

Is there some way to ensure a faster copy? I believe I've seen mentions of dd, but does this really speed up the copy or is it a placebo?

1 Answer 1

1

No, using dd isn't "placebo."

New volumes created from existing EBS snapshots load lazily in the background.

So, if you don't do something to encourage it to be not-so-lazy, EBS would naturally assume that there's no particular hurry, so it needn't tax its own resources, or those of S3, to load the data quickly.

If your instance accesses data that hasn't yet been loaded, the volume immediately downloads the requested data from Amazon S3, and continues loading the rest of the data in the background.

If you need to ensure that your restored volume always functions at peak capacity in production, you can force the immediate initialization of the entire volume using dd or fio. For more information, see Initializing Amazon EBS Volumes.

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-restoring-volume.html

You can even do this with the volume mounted. Since you're only reading from -- not writing to -- the block device, the OS will not notice or see this as a problem.

Since fio can run reads in parallel, it can warm your volume faster than dd but dd will get job done.

If you like progress meters, try pv - pterab /dev/xvdN > /dev/null. (sudo apt-get install pv first.) This will not necessarily be the fastest, but it's solid and more fun to watch, since it gives you current and sustained throughput, percent complete, and an ETA. When it's done, your volume is fully warmed up.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .