1

Two days ago I began the recovery of a 1TB failing HDD which was handed out to me with the hope that I could salvage most of it for a cheap price.

At first it behaved erratically, often being disconnected suddenly and making scary noises, with a copy speed varying between a few KB per second and about 50MB per second (it was a hot day, I tried to prevent it from overheating with a laptop cooling pad underneath and a cooling block over it, which I changed every hour or so). Then, during the first evening, it became more stable, but the average copy speed went down significantly, to around 3-4MB/s. Now, having recovered 250GB, I'm down to about 400KB/s on average, which is painfully slow (at least it doesn't seem to decline further).

So my questions are:

  • I'm doing that recovery to a NTFS partition, which, from what I read quite late in the process (on this french guide), is not recommended as it might slow down the recovery considerably. Is that (still) true, and if so, why?
    • Or is this a thing of the past, when the NTFS driver for Linux wasn't mature enough? (I'm using the latest Knoppix live DVD, copied to a memory card as it wouldn't successfully boot from a DVD-RW.)
  • Would it be worth the trouble to convert the partition to a native Linux format like Ext4 at this stage? I mean, would it significantly improve the copy speed?
    • Or is it normal to experience such a slowdown with a failing drive, after the first pass where most “healthy” sectors have already been recovered? (The SMART parameters are worsening, the “overall-health self-assessment test result” went from “PASSED” to “FAILED”, the number of reallocated sectors went from 144 to 1360.)
  • Is there something else that I can do to improve the recovery ratio and/or recovery speed?
  • Are there options in ddrescue which I could try with some real benefit?

I made the first runs with this command :

ddrescue -n -N -a500000 -K1048576 -u /dev/sdc /media/sda1/Hitachi1TB /media/sda1/Hitachi1TB.log

(The -n & -N switches supposedly bypass the scraping and trimming phases – although I'm not sure at what point in the process those actions are attempted by the program and if that is actually useful to bypass them. Then I specified a minimum copy speed of 500000 bytes per second, and a value of 1MB for “initial size to skip on read error”, attempting to copy as quickly as possible the areas which are still healthy or easy to access. The -u is for “unidirectional” : in a former recovery with another HDD, copying in reverse sense with the -R switch seemed to improve matters, but with this one it seems to wreak havoc, and it's apparently more stable with that switch.)

Now, after it completed one pass, I removed most of these parameters, only keeping the -u. I tried the -d switch at some point (“use direct disc access”), but then nothing was copied, the “error size” grew very quickly.

  • From my experience, writing to NTFS is indeed slower than ext4. – Andrea Lazzarotto Aug 26 '17 at 21:39
  • I agree with Andrea's comment: writing to NTFS is slower. But not that slow. Slowness is expected with ddrescue and a faulty drive. See this answer and compare this question. – Kamil Maciorowski Aug 26 '17 at 21:42
  • Thanks for those replies. In the 2nd linked thread, someone reports that switching from ExtFAT to Ext4 did significantly improve the copying rate (7MB/s isn't much but still 70x better than 100KB/s !). If I can hope to get a similar result by switching from NTFS to Ext4, how can I proceed to do it safely, with no risk of losing what has been recovered so far ? Can I convert the partition in place, and with which command ? (Still new to Linux environment.) I guess it would be wise to first image the NTFS partition, in case it goes awry. Then how can I access the Ext4 partition in Windows ? – GabrielB Aug 27 '17 at 4:15
  • So, to answer my own question... I did this : 1) archive the content of the NTFS partition to safeguard it 2) shrink the NTFS partition and create an Ext4 partition with gparted 3) copy files from NTFS to Ext4 4) resumed the ddrescue recovery -- actually I had to start all over again as the preview (-P switch) showed nothing but 00's, don't know why... (Very important to turn that on !) And now it's going blazingly fast with an average rate of 90MB/s ! O_o (Twice the best rate I had earlier, and strangely no error so far, 100GB in.) So I'll never attempt a recovery onto a NTFS volume anymore. – GabrielB Aug 27 '17 at 18:15
  • 1
    Formal notes: (1) Your comment above is not an answer. To really answer your own question write an answer in the answer field. I guess your final answer may be "it wasn't worth it" because now you're reading fast for the second time and later you will read just as slow as it was before you interrupted. (2) To reply to somebody else's comment with your own comment call the person like this @KamilMaciorowski, this way he or she will be notified. The author of the question/answer under which a comment is written gets notification regardless of this, so I don't need to call you this way here. – Kamil Maciorowski Aug 28 '17 at 7:04
2

To complete my comments above (sorry for the formal inconveniences / inconsistencies): I would say that it was worth it, even though I don't quite understand why. The second attempt, recovering to an Ext4 partition, had a significantly higher copying rate at the beginning (about 90 MB/s on average, whereas I only had about 50 MB/s at best for the first attempt, recovering to an NTFS partition), and no errors or even slowdowns. But then, after copying about 165 GB (so earlier than before), it became very unstable and slowed down to a crawl, made clicking and whirring noises again (it was a very hot period which didn't help – I tried to cool it down as much as possible, using a laptop cooling pad below and a freezing pack over it, changed every hour or so); I tried again and again (sometimes it got back to a 120 MB/s rate for a few seconds then back to 0), but I had to abandon it after a while.

Here is a ddrescueview map of the first recovery:
ddrescueview 1st attempt NTFS partition

There's an interesting pattern, with stripes of easily recovered data alternating with very slow or unreadable data. From what I know, it would seem to indicate that a head came in contact with a platter, damaging the surface and releasing magnetic dust, which then spread with the centrifugal force. And since the servo track (which contains essential information for the startup process) is located at the outer edge of the hard drive (it's a 3.5" Hitachi 1 TB), some of that dust may have reached it, making it difficult to access, which could explain the frequent clicking noises at startup. (Correct me if I'm wrong.)

Here is a ddrescueview map of the second recovery:
ddrescueview 2nd attempt Ext4 partition

So the hard drive became very unstable and the recovery increasingly difficult after about 165 GB, but before that the copying rate was consistently high, with no skipped areas. I later used the ddru_ntfsbitmap method, for the last attempts, so the free space was mostly skipped.

Here is a ddrescueview map of the log file created with ddru_ntfsbitmap, showing the areas of the hard drive containing actual data in green, and free space in grey:
ddrescueview ddru_ntfsbitmap log file

Luckily, most of the actual data was located in the first quarter and was successfully recovered. Now I have yet to combine the good parts of those two images, and extract the actual files, probably with R-Studio (best data recovery software that I have tried).


One thing I later found out which is interesting and peculiar, regarding my initial question (I guess I should have put this as a comment, as per the formal rules, but it would have been too long and I couldn't have provided screenshots).

I attempted to copy the rescued areas of image 2, on the Ext4 partition, which were missing in image 1, to that image 1, on the NTFS partition{1}, which should have been done at a very high rate (input and output being on a healthy 2 TB HDD), yet I got a speed of, guess what, only 660 KB/s on average!

Command used (log file for image 2 used as a domain logfile):

ddrescue -m [image2.log] [image2] [image1] [image1.log]

Screenshot:
ddrescue copy rescued areas image 2 to image 1

So I stopped and did the opposite: I copied the rescued areas in image 1 (NTFS) which were missing in image 2, to that image 2 (Ext4) — and now the copying rate was about 43000 KB/s or 43 MB/s on average (maybe slightly slower than what should be expected for a copy on the same HDD, for a Seagate 2 TB which has a max write speed close to 200 MB/s, so should be able to reach about 100 MB/s for a copy from one partition to the other, but still almost 100× better than the first attempt). What would be the explanation for such a tremendous discrepancy?

Command used:

ddrescue -m [image2.log] [image2] [image1] [image1.log]

Screenshot:
ddrescue copy rescued areas image 1 to image 2

I noticed that the image files on both partitions had a “size on disk” {2} corresponding to the amount of data which had been actually written, very far from the total size (1 TB or 931.5 GB), even though I didn't use the -S switch (“use sparse writes for output files”). Image 2 (after being completed with extra rescued areas from image 1) has a “size on disk” of 308.5 GB, while image 1 has a “size on disk” of 259.8 GB. Could it be related with the slow copy rate, if the Linux NTFS driver somehow has trouble dealing with sparse writings? And how come the whole size was not allocated as soon as sectors at the end were written, considering that I did not use that -S switch?

I tried to use the -p switch (“preallocate”) at the very beginning of the process, thinking that it would be “cleaner”, more straightforward, easier to deal with in case something would go wrong (if the recovery needs to be recovered...), but I had to stop as it was way too long and I wanted to get started ASAP. Then I figured that by using the -R switch (“reverse”) temporarily, it would write the very last sectors to the output file, thus allocating the full size as I intended; it indeed resulted in increasing the size of the output file to 931.5 GB, but the “size on disk” was in fact way lower (I noticed it later when accessing the HDD used for that recovery on Windows and seeing the abnormally high amount of free space).
________________
{1} I still don't understand how the second recovery attempt could produce a so much better outcome for the first 100 GB or so, despite the fact that the health status of the HDD had declined in the meantime.

{2} By the way, the word “disk” should be replaced, on Windows and Linux systems alike, as there are data storage units which are not “disks”...

  • I'm re-reading this thread a year later : what I wrote about the “stripes” pattern is most likely wrong – from what I learned later on, this is observed when one or more head(s) of the drive start(s) failing (since the data is recording in alternating “strokes” of a few GB on each platter's surface, the size of each “stroke” being fixed for each model). – GabrielB Nov 24 '18 at 19:01
  • Regarding the success of the recovery, only about 120 files from the owner's personal files were damaged, and since there were many duplicates I managed to repair about 100 of them (in several instances two duplicates were corrupted in different places so I could fill the holes in one with the other using WinHex), leaving about 20 damaged files. An excellent result for a software-only recovery, considering that the drive was clicking like crazy when I got it ! – GabrielB Nov 24 '18 at 19:03
-2
  1. You might want to copy the disk image first with dd command

    sudo dd bs=[block_size] count=[NofBlocks] if=[in_file] of=[out_file]

where

[in_file] - can be the broken disk, say /dev/sdd2

[out_file] - location of the output image file.

  1. Mount the image and try to recover it.
  • 1
    -1. The question is about NTFS vs ext4, this answer doesn't answer the question at all. Besides ddrescue was created to deal with faulty devices; with dd you can use conv=noerr but still ddrescue with its log file support and specialized options is a way better choice in this case. – Kamil Maciorowski Sep 11 '17 at 20:31
  • First you have to create an image, the rest is irrelevant. "ddrescue very slow, writing to NTFS" is very strange claim, it has to be faster as it skips bad blocks, unless the NTFS is Not mounted properly, test it first and only then try to recover on it. – Alex Sep 12 '17 at 21:08
  • @Alex Well, did you read what I wrote ? The copying rate when writing a new file from the Ext4 partition to the NTFS partition is about normal (a bit slow but not excruciatingly so) : I copied the 2nd recovery (with ddrescue and -S switch this time to save space) from Ext4 to NTFS (both on the healthy 2TB HDD) with an average of 30-40MB/s. But before that when I attempted to complete the 1st image file already in place (and probably fragmented), with extra recovered areas from the 2nd image file, the rate went down to about 0,6MB/s. What would “not mounted properly” mean in this case ? – GabrielB Sep 13 '17 at 15:50

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.