I'm trying to recover files on a dying hard drive, using GNU's ddrescue. The drive is rather large (1TB), and all I have to store the image is another 1TB drive.

ddrescue fails near the end, with an error about disk space, even with the -S option.

Why isn't it possible to compress the image as it is created ? It is possible with dd and dd_rescue.

Also, man ddrescue says that the -S option doesn't work on all systems, but how can I know if it works on mine ?

  • 1
    Is there a reason you can't just ddrescue [...] | gzip -9 > /path/to/destination/file? – Aaron Miller Aug 12 '13 at 14:21
  • 1
    I've not tried this, but I wonder if getting a single larger drive was not an option, you might be able to get a second disk, create a larger LVM than your source disk, and store the image there. – Journeyman Geek Aug 12 '13 at 14:30
  • 1
    @AaronMiller Yes: ddrescue requires a destination with seek support, due to its mode of operation (cf. JourneymanGeek's answer). Therefore, it cannot output to pipes. – Bob Aug 12 '13 at 14:38

gddrescue dosen't image the file in order - it goes back and retries, and fills in the blanks, and I suspect this is why you can't effectively pipe it into something else. The man page goes into that in detail

GNU ddrescue manages efficiently the status of the rescue in progress and tries to rescue the good parts first, scheduling reads inside bad (or slow) areas for later. This maximizes the amount of data that can be finally recovered from a failing drive.

The standard dd utility can be used to save data from a failing drive, but it reads the data sequentially, which may wear out the drive without rescuing anything if the errors are at the beginning of the drive.

Other programs switch to small size reads when they find errors, but they still read the data sequentially. This is a bad idea because it means spending more time at error areas, damaging the surface, the heads and the drive mechanics, instead of getting out of them as fast as possible. This behavior reduces the chances of rescuing the remaining good data.

Its by design, and the solution, unfortunately, is to get a bigger drive. For the -S argument to work, I believe the used space on the source drive must be smaller than the destination drive.

  • 1
    A bigger drive might not be necessary. There are some filesystems and other modules/programs that can provide transparent compression. Really, ddrescue should work just fine with any seekable destination file, which can be compressed. – Bob Aug 12 '13 at 14:40
  • I consider lots of big drives essential for recovery and forensics. Else its just to painful with every other variable we have on hand - especially messed up drives. – Journeyman Geek Aug 12 '13 at 14:45
  • 3
    If it's your job, sure. But for a one-time thing, where you might be able to just sneak past, sometimes the purchase of a new drive is not desirable. Also, everything critical is backed up. Right? Right? (Seriously, @Manu, back up everything you can't afford to lose. Storage can and will fail.) – Bob Aug 12 '13 at 14:49
  • Yes, lesson learned :) I have a NAS and a cloud based solution, but one was too old, the other did not copy everything. I should have checked sooner. – Manu Aug 12 '13 at 16:58

It is possible to compress images created with ddrescue on-the-fly. Thing is, ddrescue requires a seekable destination, since it will go over in several passes (and therefore must have the ability to jump backwards to fill in earlier gaps, as @JourneymanGeek explains in his answer). This means you cannot use a pipe as the output, since a pipe is not seekable. Therefore you cannot pipe to compression programs.

One way to get around this is to use transparent compression. Some filesystems (notably, Btrfs, amongst others) provide this built-in. Alternatively, you can use filesystem drivers to provide transparent, seekable compressed storage, e.g. fusecompress.

  • 2
    Of course, compressibility of your source data can vary a lot. Especially since this is a drive image, which will not see 'free space' (very compressible) - it will actually copy whatever was in the blocks, whether they were 'deleted' or not. Also, the level of compression will vary across different methods used, and transparent compression often sacrifices size for speed, though that may be adjustable. – Bob Aug 12 '13 at 14:51
  • 1
    I have thought about compressed file systems, but I have no experience at all with them. – Manu Aug 12 '13 at 16:56
  • @Manu The fusecompress one looks fairly simple. – Bob Aug 12 '13 at 17:56
  • brashear.me/blog/2017/11/14/… has instructions for using btrfs+lzo (I'm going to need to do something like this, since I ran badblocks on the drive and stopped it while it was filled with 0xAAAAAAAAA..., then wrote data to it, then it died and I'm trying to recover it. So it won't sparsify the output file because all the empty space is 0xAAAA instead of 0x0000. 🙄) – endolith Jan 23 '20 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.