0

I have an HDD with 10319 bad sectors(reallocated sectors), this is the SMART information of the drive; There are 3097029168 sectors, bad sectors/total sectors is 0.00000333190274945450562188570256307, I would like to keep using it until the reallocated sector count increases. enter image description here

From what I gathered, the bad sectors are marked as bad and reallocated into G-List(Growing Defect List), when the operating system tries to access the sectors in G-List, the read/write requests are redirected automatically to good sectors located in a spare area inaccessible to the OS otherwise by the hard disk controller, the redirection causes delays, decreases performance.

However I know hard drives, HDD and SSD alike, are "born" with defects, because modern manufacturing technology isn't perfect, the manufacturers do a remapping before the disks are shipped, and allocated the defects into P-List(Primary Defect List), and P-List sectors do not affect performance.

I am wondering, is there a way to renumber the sectors, in such a way that the bad sectors don't have assigned addresses, so that the bad sectors are impossible to access or even be listed, effectively making the disk bad sector free, just with smaller size? Or can the G-list sectors be moved into P-list?

For example, using logical block addressing(LBA), if there are a total of 10000 sectors, the sectors with LBA's 42, 256, 365, 1024 are bad, after renumbering, sectors in LBA range 0-41 will remain unchanged, sectors in LBA range 43-255 will be sectors 42-254 (-1), sectors in LBA 257-364 will be 255-362 (-2), sectors in LBA 366-1023 will be 363-1020 (-3), sectors 1025-9999 will be 1021-9995 (-4), so that there will be 9996 continuous good sectors? There must be a way to read G-List information, is there any method to achieve this so that I will have a good disk with 3,097,018,849 continous good sectors(provided it doesn't deteriorate)?

Update: I had found this:

Once the drive layout is defined in the manufacturing process the search for defects can occur. Keep in mind that this is all occurring before the definition of the Logical Block Addressing scheme is applied. The “P” list consists of identified physical locations of “Bad” sectors on the hard disk drive. This list is then used to assist in the Logical Block Addressing of drive space for the writing of data. Here is the trick…During manufacturing, the layout of the LBA blocks occurs by shifting the LBA block sequence past the “Bad” sector. The LBA layer simply skips over the identified “Bad” location in the “P” list and continues sequentially in LBA order. The “P” list is a list of “skipped” physical locations on the hard disk drive platters.

https://www.forensicfocus.com/articles/forensic-imaging-of-hard-disk-drives-what-we-thought-we-knew-2/#:~:text=The%20physical%20geometry%20layout%20of,by%20cylinders%2C%20heads%20and%20sectors.&text=The%20%E2%80%9CP%E2%80%9D%20list%20is%20a%20list%20of%20%E2%80%9Cskipped%E2%80%9D,on%20the%20hard%20disk%20platter.

It seems I can skip the G-List sectors by moving them into P-List, and redefine Logical Block Addresses to skip P-List, how can I do it?

Edit: I had found out I can achieve this with PC-3000, but I can't afford it, does there exist any free and open source alternatives to PC-3000?

Edit1: It seems I was right, the P-List is a skip list, they don't have LBA's, the process described above happened in factory during assigning LBA's, and I had already copied all important data that can't be downloaded online off the disk successfully without errors and then already wiped my disk clean with these:

diskpart /s %userprofile%\desktop\wipedisk.txt
format G: /fs:NTFS /a:4096 /x /p:1 /y
format G: /fs:NTFS /a:4096 /x /q /y
pause

file wipedisk.txt

select disk 1
clean all
create partition primary
format fs=ntfs
assign letter=G
quit

I am serious I want to move G-list into P-list.

Update: I had just found out the technical term for my intended operation, and confirmed low-level format can do it:https://en.wikipedia.org/wiki/Sector_slipping

9
  • My disk is Seagate Expansion 2TB, disk with enclosure firmware revision 0708, disk only model number ST2000LM007- 1R8174, firmware revision SBK2, how can I move the bad blocks into P-List? – Xeнεi Ξэnвϵς Dec 25 '20 at 11:40
  • There must be a way to do this, because this is what manufacturers had accomplished countless times, I know this is vendor specific, and I likely need firmware tools, but I think someone must have had reverse engineered it. – Xeнεi Ξэnвϵς Dec 25 '20 at 11:48
  • 2
    A simple low-level format will do the trick, no firmware hacks necessary. It is pointless, though. – Eugen Rieck Dec 25 '20 at 13:19
  • 1
    Does this answer your question? Can a disk with bad sectors be trusted after the bad sectors are fixed? – Hannu Dec 25 '20 at 14:04
  • 2
    diskpart does not at all do any sort of low-level format. It deals with partitions and file systems, both of which are way up higher in the stack. Unfortunately in Windows the command format is badly named. The Unix name of make filesystem is much more to the point. – Eugen Rieck Dec 25 '20 at 14:17
3

First of all: It is untrue, that primary defects impose no performance penalty. It is a matter of statistics, that the likelyhood of needen a track change during a read operation over sequential sectors increases, when the number of sectors on a track decreases.

Now: The primary defect list is created before any data is stored on the drive, it is a skip list, while the grown defect list is maintained with data already on the drive, so it is a redirect list. If you were to convert the entries for GB-Sectors into entries for PB-Sectors, you would lose that redirection information and need to reformat the drive. With a drive already in such a bad shape I assume, that the procedure of moving everything off, then reformating, then moving everything back on again would most likely create so many new GB-Sectors as to make it pointless.

With 0.00033% of sectors reallocated you see a likelyhood of again ca. 0.00033% to hit one, so you will suffer two repositions every ca. 3000 reads without taking into account any caching. I do not think, that the performance increase after caching would be appreciable.

EDIT

As requested in the comments, here is a short how-to on low-level formating: Since I don't know, if Windows can do this reliably (or even at all), the easiest way is to boot from a Linux USB stick, if you still use Windows.

  • Step one: Identify the disk. This is easy for an USB disk, as by booting with it unplugged, then observing /var/log/syslog will tell you easily, which disk it is (e.g. /dev/sdb).
  • Step two: Do the format. First install scsitools (sudo apt install scsitools und dpkg flavours), then just run scsiformat -t 5 /dev/sdX with the disk you identified in the first step.
8
  • I had already moved everything that can't be downloaded I can move out from the drive, and did diskpart clean all and format G: /fs:NTFS /a:4096 /x /p:1, now the disk is empty, and I would really like to move G-List into P-List; BTW, I had upvoted your answer, you had one upvote and now you have two. – Xeнεi Ξэnвϵς Dec 25 '20 at 12:02
  • Formating isn't ment in the sense of creating a file system (such as Windows format command), but as low-level format, the process that creates the structures like sectors. I don't know if WIndows can do that, it's easy with sgutils on Linux. This will recreate the PDL from scratch, including any newly defective sectors. – Eugen Rieck Dec 25 '20 at 12:24
  • There is no other reason to ask this question then for fraudulous intentions. You describe yourself as "I am trying to be a hacker like those in Hollywood movies, hackers that can hack everything." You don't become a hacker by going from one forum to another asking that same question again, basically asking for solutions. A lot of your questions could be answered by trying things out personally on your machine. You are searching for ready-made scripts. You won't become a hacker that way. Intelligence paired with curiosity and endurance might get you there - not your current way! – r2d3 Dec 25 '20 at 17:50
  • @r2d3 I was unaware, that I describe myself this way - and it is unlikely, given my age. Or did you intend to post your comment on the question, not the answer? – Eugen Rieck Dec 25 '20 at 20:43
  • Eugen, I apologize for the misunderstanding I caused, my post ist referring to Xeнεi Ξэnвϵς, sorry! – r2d3 Dec 25 '20 at 23:54
2

No one has bothered to reverse engineer the firmware, because it is pointless.

The installed firmware is digitally signed with the signature of the manufacturer, so it is impossible to replace with your own version. (I don't know if it's also encrypted, but that's certainly possible.)

This is an important security measure that should reassure you that the disk you buy has not been infected with viruses on the way to your computer.

2
  • Err, there are a number of tools commercially available for reverse engineering and modifying disk firmware. For examples, look up negative sector/cylinders, ACE Laboratory, PCS-3000. And disk firmware is not one single blob but a number of firmware modules. – fpmurphy Dec 26 '20 at 5:44
  • @fpmurphy: There is a difference between the disk-resident modules and the core system (which is often chip-resident). – harrymc Dec 26 '20 at 6:50
1

I am wondering, is there a way to renumber the sectors, in such a way that the bad sectors don't have assigned addresses, so that the bad sectors are impossible to access or even be listed, effectively making the disk bad sector free, just with smaller size?

That is exactly what the firmware in all drives with "reallocated sectors" is doing: The logical block address (LBA) is remapped transparently, and the OS doesn't notice any difference. In fact, the OS cannot access any of the bad sectors ("the bad sectors don't have assigned addresses"), nor the spare sectors (so if you want, those don't have assigned addresses, either).

So to the OS, it looks like the disk is bad sector free. Many OS do have provisions for bad sector lists that come from a time where the firmware didn't reallocated sectors, but with a firmware that does (and basically all modern harddrives do), those OS-level bad sector provisions have become useless.

(Though people are still frequently worried about bad sectors, and keep asking here what they need to do about them. And the answer is: nothing)


But the bad sectors in G-list are merely redirected,

I am not sure what your understanding of "they are merely redirected" is. They are "merely redirected" in the same way that the P-list "merely redirects": for the OS they don't exist, and can't be accessed.

the redirection decreases performance

In the sense that "a read of a sequence continuous sectors is now no longer continuous, and requires head movement", yes. Not in the sense "the redirection takes time to perform", that is negligible. So with a random access pattern, there's is no difference in performance.

And the P-list causes the same performance issue.

and the bad sectors can be directly accessed when doing full disk operations like full format, clean all..

They won't be accessed. The firmware keeps ignoring them.

And I/O to bad sectors, redirected or not, will likely make more bad sectors...

Why should it? There are no longer any bad sectors, as far as the OS is concerned. You can't make "more bad sectors" by only accessing good sectors. (And you can't even make "more bad sectors" by accessing bad sectors).

I want to move the bad sectors in G-list to P-list so that they don't get counted by smart 05 rsc,

You can't (unless you reverse engineer the firmware), and even if you could, you wouldn't get anything out of it.

and the number of total sectors displayed will decrease by the number of bad sectors,

That's not necessary: The number of spare sectors (that's never shown anywhere) already decreased when the sectors were reallocated.

Example: In the beginning your disk showed a capacity of 1000 sectors, and it had 50 spare sectors, for a total of 1050 "only visible to firmware" sectors. Over time it found 20 bad sectors and reallocated them. Now it has 20 reallocated spare sectors, 30 available spare sectors, and still shows a capacity of 1000 sectors.

Your desire to do something here comes from not understanding how the firmware works. The firmware already does what you want to do manually.

to all softwares they officially don't exist,

To all software the above 50 spare sectors, no matter if reallocated or not, already offically don't exist. All software can access exactly 1000 sectors on the hard disk.

no new bad sectors will ever be created by reading/writing to them

As the bad sectors can now no more longer be accessed within in the 1000 "reported" sectors, nobody can read or write them.

4
  • You are right, it's a good answer indeed, but every drive has a P-list hidden from SMART 05 Reallocated Sector Count the moment they are "born", they are skipped when assigning LBA's, for all intents and purposes, they don't exist, they can never be accessed; But the bad sectors in G-list are merely redirected, the redirection decreases performance, and the bad sectors can be directly accessed when doing full disk operations like full format, clean all...And I/O to bad sectors, redirected or not, will likely make more bad sectors... – Xeнεi Ξэnвϵς Dec 25 '20 at 13:38
  • I want to move the bad sectors in G-list to P-list so that they don't get counted by smart 05 rsc, and the number of total sectors displayed will decrease by the number of bad sectors, to all softwares they officially don't exist, so no new bad sectors will ever be created by reading/writing to them, so rsc will be 0 unless something else is actively making more bad sectors, then I will replace the disk. – Xeнεi Ξэnвϵς Dec 25 '20 at 13:47
  • " and the number of total sectors displayed will decrease by the number of bad sectors" Where do you think is a decreasing number of total sectors displayed? – r2d3 Dec 26 '20 at 0:14
  • The number of total sectors is not decreasing as bad sectors are remapped. That's exactly the point. Invisible spare sectors that are not counted in the total because they are "spare", and not normally accessible, are used instead of the bad sectors, which are now inaccessible after remapping. – dirkt Dec 26 '20 at 5:02
0

First:
2TB disks appeared on the market around January 2009.
20TB disks were previewed in June last year (2019).

If doubled capacity is a single generation shift, then that is a difference of four generations maybe even close to five.
Counting three-year periods; it is at least three generations.

Why bother with so old hardware?

Second:
There likely is a difference between production imperfections, and faults (bad sectors) that appears after the disk has been in use for some time.

The new faults may be caused by e.g.

  • technology dependent causes,
  • degradation of a magnetic surface,
  • aging of the material or by
  • mechanical wear

... where at least the last is likely to escalate with time and use.

In the end:
Judging the risk of further damage and the wish to retain ALL data in a safe manner makes me go with the only safe alternative;

Replace the disk and gain somewhere near ten times the data storage space.

3
  • Good suggestion, but buying a new one requires a big sum of money, it'll cost me substantially. By the way, next time when I buy a disk, I will buy a 3TB one, maybe 4TB, I don't need 20TB storage space. – Xeнεi Ξэnвϵς Dec 25 '20 at 14:11
  • Weigh that against losing a good amount of data. In my opinion there is no other viable alternatives. – Hannu Dec 25 '20 at 17:34
  • @XeнεiΞэnвϵς When you say, “Good suggestion, but buying a new one requires a big sum of money…” Honestly 2TB SATA hard disks are very inexpensive nowadays. Many times as low as $50 and as high as $100. Regardless, the idea you want to adjust a hard drive’s firmware to magically make a dying drive somewhat useful is ridiculous. You will be spending more time than necessary to extend the life of a drive to a few days or a week. Just get a new drive. – Giacomo1968 Dec 26 '20 at 3:46
-1

Do not answer this question - it will benefit a few at the expense of many!

Technically there is no reason why there are two lists but there is a reason to show broken sectors arising during lifetime and to prevent users from deleting this important diagnostic information.

With the intended manipulation of the author you can easily lure people intent believing that a broken disk is still good.

allocated the defects into P-List(Primary Defect List), and P-List sectors do not affect performance.

This is just a pretext. Looking up a spare sector number in a list of n broken sectors costs log(n) steps in a tree structure. This applies to both lists, assuming they are both maintained in a tree structure.

This is outright bad - somebody who wants information about odometer manipulation.

In a comment you state:

But the disks were made with defects, manufacturing imperfections are inevitable, then in your logic, all disks should be broken, there would be no usable disks!

Where do I state that? Which logic implies that? Please explain!

2
  • But the disks were made with defects, manufacturing imperfections are inevitable, then in your logic, all disks should be broken, there would be no usable disks! Also the sectors in P-list don't have Logic Block Addresses therefore never remapped to spare area, they only have a Physical Block Address known only to the firmware so nothing except firmware and softwares that can access firmware can access them, everything else wouldn't even know they exist, including SMART, and I would decrease max LBA so that no reserved sectors will be used... – Xeнεi Ξэnвϵς Dec 25 '20 at 22:29
  • There is no way to do what the author wants without reversing the firmware, anyhow. If someone can modify the firmware, all bets are off: one can make an utterly broken disk have good SMART values (which isn't even needed, there are stores successfully selling returned failed harddisks as "new", as you can see from questions here). And reverse engineering harddisk firmwares is time consuming, difficult, and will have to be done for each harddisk model, so it's not really a threat... – dirkt Dec 26 '20 at 5:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.