5

I have a 3TB HDD and Ubuntu 14.04 live DVD is showing it as being 746.5GiB. I've seen this sort of problem with other disks in other situations before and never found a solution. I've even had Seagate replace a HDD telling me they were unable to fix the HDD after a disk imaging tool somehow permanently told the HDD it was a smaller disk.

Two questions:

  • How did this happen?
  • How do I fix it?

HDD Background for this case

  • The disk used to be part of a ZFS RAIDZ using the bare disk instead of a partition.
  • it sat on a shelf for a quite a few months.
  • I used GParted to try and delete everything on the disk

I'm currently running the command

dd if=/dev/zero of=/dev/sdd bs=16M

Before that I ran:

root@ubuntu:/home/ubuntu# dd if=/dev/zero of=/dev/sdd bs=10M count=128
128+0 records in
128+0 records out
1342177280 bytes (1.3 GB) copied, 42.8214 s, 31.3 MB/s
root@ubuntu:/home/ubuntu# smartctl -i /dev/sdd
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-32-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Caviar Green (AF, SATA 6Gb/s)
Device Model:     WDC WD30EZRX-00DC0B0
Serial Number:    {blanked}
LU WWN Device Id: 5 0014ee 0036bc22d
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 1.5 Gb/s)
Local Time is:    Sun Jul 17 07:44:41 2016 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
root@ubuntu:/home/ubuntu# gdisk -l /dev/sdd
GPT fdisk (gdisk) version 0.8.8

Partition table scan:
  MBR: not present
  BSD: not present
  APM: not present
  GPT: not present

Creating new GPT entries.
Disk /dev/sdd: 1565565872 sectors, 746.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 1F876634-0284-4A1C-8FDF-34A255B9DCCC
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1565565838
Partitions will be aligned on 2048-sector boundaries
Total free space is 1565565805 sectors (746.5 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
root@ubuntu:/home/ubuntu# 

It is connected via a USB HDD dock. Perhaps this is the source of the problem. I'll rewire so it is plugged in directly to a SATA port on the motherboard. In the meantime here is the extra info requested.

root@ubuntu:/home/ubuntu# sg_readcap --16 /dev/sdd
READ CAPACITY (16) not supported
root@ubuntu:/home/ubuntu# sg_readcap  /dev/sdd
Read Capacity results:
   Last logical block address=1565565871 (0x5d50a3af), Number of blocks=1565565872
   Logical block length=512 bytes
Hence:
   Device size: 801569726464 bytes, 764436.5 MiB, 801.57 GB
root@ubuntu:/home/ubuntu# hdparm -N /dev/sdd

/dev/sdd:
 max sectors   = 5860533168/1(5860533168?), HPA setting seems invalid (buggy kernel device driver?)
6
  • 2
    Apparently because 3000592982016 / 512 = 5860533168 = 0x15d50a3b0 and it got truncated to 0x5d50a3b0 = 1565565872. How is it connected to your computer?
    – Tom Yan
    Jul 17, 2016 at 10:29
  • 1
    Get sg3_utils (package named sg3-utils in Ubuntu) and run sg_readcap -16 /dev/sdd. If it reports the capacity correctly, then probably your gdisk is too old to handle a 3TB disk. Otherwise it would be either the kernel, or your USB enclosure/docking/adapter.
    – Tom Yan
    Jul 17, 2016 at 10:47
  • What is the output of hdparm -N /dev/sdd? Jul 17, 2016 at 11:41
  • This HPA setting seems invalid is worth investigating. Check similar case I have found; Ultimate Boot CD allegedly helped. Jul 17, 2016 at 12:42
  • 2
    Well problem solved. The USB docking does not support SCSI READ CAPACITY (16) command, which means when it translate the capacity from ATA IDENTIFY DEVICE data, it can at most report a 32-bit capacity by responding to a SCSI READ CAPACITY (10) command, while 0x15d50a3b0 requires 33 bits. See SBC for details if desired.
    – Tom Yan
    Jul 17, 2016 at 16:32

1 Answer 1

11
root@ubuntu:/home/ubuntu# sg_readcap --16 /dev/sdd
READ CAPACITY (16) not supported

This means when your USB docking translate the capacity from the drive's ATA IDENTIFY DEVICE data (seen in hdparm -I / smartctl -i), it can at most report a size up to 32-bit (i.e. 0xffffffff, 4294967295) in terms of number of logical sectors. This is an inherit limitation of SCSI READ CAPACITY (10):

Logical Sector Size | Maximum capacity supported (TiB / TB)
         512        |              ~2.0 /  ~2.2
        4096        |             ~16.0 / ~17.6

Since your drive is an AF 512e drive that has totally 5860533168 / 0x15d50a3b0 512-byte logical sectors, which requires 33 bits to represent, only a SATA/USB bridge that supports SCSI READ CAPACITY (16) can handle it properly. When the size is truncated to 32-bit, it turns from:

101011101010100001010001110110000 (5860533168)

to

 01011101010100001010001110110000 (1565565872)

The Linux kernel, or probably all OSes, will basically never issue ATA IDENTIFY DEVICE command "directly" (i.e. encapsulated in a SCSI ATA PASS-THROUGH command) to USB drives, but SCSI READ CAPACITY commands (which you issued manually with sg_readcap), to get the capacity of them.

Only when the drives are actually SATA drive connected with a SATA/USB bridge, the command will be handled by the SCSI-ATA Translation Layer implemented in the bridge, which will then issue ATA IDENTIFY DEVICE command to the SATA drive to get the information it needs to form the response data for the READ CAPACITY command.

But utilities like hdparm and smartctl are (almost) exclusively for ATA drives, so they pretty much do everything with ATA PASS-THROUGH. (Also because they are userspace utilities, it is expected you, the user, will only use them on the appropriate type of devices.) That's why you end up getting different capacity in different places.

4
  • That's a great answer but... can the problem be somehow worked around? I mean, is there a way to persuade the kernel (or some partitioning tool) to use the correct size? I have a similar problem and it'd be a pity to have to replace an USB enclosure if the only problem is that it's not reporting the capacity correctly.
    – a3nm
    Apr 26, 2020 at 22:22
  • Persuade is the wrong word. It simply couldn't get the correct size. There's no facility written in the usb storage driver to get the size of a drive with ATA IDENTIFY DEVICE in a SCSI ATA PASS-THROUGH. But it's Linux, you can always write your own hack, if you can. (There's already facility, known as quirk, for you to flag a drive and do something different for it.) And no, it's not a pity, not in my opinion, broken things simply belong to the trash.
    – Tom Yan
    Apr 27, 2020 at 3:19
  • Thanks for the pointer! I agree it's simpler to just trash things as soon as they have a problem, but this philosophy doesn't look environmentally sustainable given our planet's current trajectory, so I don't think it's an unreasonable question to see what can be done with the stuff I have :)
    – a3nm
    Apr 27, 2020 at 8:30
  • I don't understand. It's only a Linux issue. The problem doesn't show up on my Win7. I asked another question but now it's hidden because someone thinks it's duplicate to this one. If it is, why can Win7 handle 3T as disk as being 3T but Linux not?
    – Catscarlet
    Jun 26, 2020 at 1:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.