Merging 2 ddrescue output of the same drive efficiently

Question

I am trying to rescue a partition in a failing 500GB hard drive at /dev/sdb using ddrescue to a second 1TB drive. The ddrescue command I use for first pass is:

ddrescue -n /dev/sdb2 core.img core.log

followed by second pass:

ddrescue -d -r 3 /dev/sdb2 core.img core.log

The problem is, I accidentally mistype the image name for the second command. Instead of core.img, I write core.ing and I did not realize it until the next morning where the second pass has run its course, and core.log overwritten to report a 100% success.

Now I am stuck with 2 massive files. Re-running the entire chore is not feasible as I am still rescuing sdb3. I already checked with hexedit and the 2 file compliments each other (i.e. if in offset 0x5000 to 0x7000 core.img is filled with zero, then the same offset in core.ing is filled with the real data, vice versa.)

I am thinking of dd-ing the core.img to /dev/sdc2 (the replacement drive for sdb I prepared) and then somehow copy only the non-zero bytes in core.ing to /dev/sdc2 but I am not sure how can I only copy the nonzero bytes, and using what, as the solutions I can think of are very cumbersome and will take forever to complete.

For reference, I am running Gentoo-based SystemRescueCD 4.9.6, sdb2 is NTFS.

Answer 1

How can I merge two different rescues?

(This assumes GNU ddrescue)

tl;dr:

1. Generate a mapfile/logfile for the first copy: ddrescue --generate-mode infile outfile mapfile

2. Create a merged copy from 2nd → 1st as described in a post to the ddrescue mailing list: ddrescue -m logfile2 image2 image1 logfile1

---

1. Generate a log file for the first image

This is slightly complicated as you effectively don't have a log file for the first pass. However, you can generate one:

When ddrescue is invoked with the --generate-mode option it operates in "generate mode", which is different from the default "rescue mode". That is, if you use the '--generate-mode' option, ddrescue does not rescue anything. It only tries to generate a mapfile for later use.

...

Don't despair (yet). Ddrescue can in some cases generate an approximate mapfile, from infile and the (partial) copy in outfile, that is almost as good as an exact mapfile. It makes this by simply assuming that sectors containing all zeros were not rescued.

... you can generate an approximate mapfile with this command:

ddrescue --generate-mode infile outfile mapfile

(emphasis mine)

from the GNU ddrescue manual; section 12, 'Generate mode'.

So you can do this for the first image (which I suggest renaming to avoid confusion, eg core-1.img):

ddrescue -G /dev/sdb2 core-1.img core-1.log

"I'm still reading from /dev/sdb, will generating a log interefere with that?"

Note that ddrescue reconstructs from the outfile (in this case, core-1.img) primarily, with very few reads from infile. I tested this with inotifywatch:

$ inotifywatch 840-linux.img # infile
$ inotifywatch 840-linux2.img # outfile
$ inotifywatch 840-linux2.log
$ ddrescue -G 840-linux.img 840-linux2.img 840-linux2.log

total  close_nowrite  open  filename
6      3              3     840-linux.img
17467  17465   1              1     840-linux2.img

total  access  modify  close_write  close_nowrite  open  filename
196    1       189     2            1              3     840-linux2.log

So it is safe to run this while another process is running on sdb (in OP's case, another rescue attempt on a different partition) as the reads are negligible.

2. Merge the two partial copies

This sort of situation has cropped up before:

So now I have two partial images of the drive that mostly don't overlap... and two matching log files to go with them that define the good areas and slow areas skipped.

Fortunately, they can be merged using --domain-mapfile:

--domain-mapfile=file
   Restrict the rescue domain to the blocks marked as 
   finished in the mapfile file. This is useful for 
   merging partially recovered images of backups, or if 
   the destination drive fails during the rescue. Use 
   '-' as file to read from standard in`

And the solution to the similar problem involved that option:

you can merge the images by typing:

cd dir1
ddrescue -m dir2/logfile dir2/image image logfile

This creates the files dir1/logfile dir1/image with all the data currently rescued. Then you can continue the rescue for example like this:

In your case, assuming you have generated a log file for core.img and labelled them 1, and kept the core.ing and core.log as 2 (lest there be more confusion!):

ddrescue -m core-2.log core-2.img core-1.img core-1.log