3

I have a 1 TB hard drive that consists of one NTFS partition which I use to back up my data (no operating system). The size of all the data in it is 726 GB, size on disk: 728 GB, and the used space when I check the properties is 731 GB. There's a 5 GB difference between the size and the used space.

  • Why is that huge difference there?
  • What's the difference between these sizes? (size, size on disk, and used space)
  • Is there a way to calculate the difference, and be sure the HDD is not messing around?
  • Is that normal?

3 Answers 3

4

It's normal. Quick answer:

Size of all the data: sum of the actual bytes of your files

Size on disk: In addition to the above, space waste by clusters/allocation units used by your files. Disks are always divided in small chunks, often 1k to 4k. So a small 0.1K file takes up a full chunk no matter what. So there's always a small loss at the trailing of every file, hence the added size (especially with lots of small files).

Used space: In addition to the above, space taken by the file system. Contains the partition table (dictionary of file locations), journal/log (to prevent corruption in case something crashes during a file operation, a notable feature of NTFS over FAT32), folders, user access permissions, creation/modification/access dates, etc, which must be recorded somewhere too.

7
  • So 5 GB?? Isn't that quite much for tables and file system stuff??
    – Ramy
    Dec 5, 2012 at 4:41
  • User access permissions? Could you explain that please? Are they somehow stored in the filsystem itself?
    – terdon
    Dec 5, 2012 at 4:41
  • @terdon Yes they are. See en.wikipedia.org/wiki/NTFS#Internals. Explains for instance why you are not considered the owner of the files if you plug a HDD in a new system or resinstall Windows (since the associated user doesn't exists anymore)
    – mtone
    Dec 5, 2012 at 4:48
  • 1
    @Ramy Well it's 2GB for cluster waste, and 3GB for the filesystem, not 5GB. I've added to my answer that it also contains creation/modified/access dates for every file, and probably other metadata. I agree that 3GB is still a lot, but I don't know the specifics.
    – mtone
    Dec 5, 2012 at 4:54
  • what specifics you need to know ??
    – Ramy
    Dec 5, 2012 at 4:57
1

This is normal. When the disk is formatted, the space is divided into blocks called sectors. When you create a file, it allocates an entire sector, not the exact amount of the file's size. Dependent on the size of the file versus the sector, you may end up using lots of extra space. Let's say you have a 512K sector size (the older drives use this) and you have a file size of 256k. When you create the file, it must use the entire sector so your file size is 256k, the space used is 512k. The difference is 256k or another way of looking at it is that you have a utilization of 50%. So you can see, it depends on your sector size, your file size, the number of files... You will almost never get an agreement of file size versus disk space used.

1
  • So the actual comparison should be between the size on disk (not the size) and the used space, that represents what's in fact the occupied space??
    – Ramy
    Dec 5, 2012 at 4:45
1

There's nothing wrong with that. As said, size of all the data is the total number of actual bytes of your files, and size on disk is size actually consumed. But more on that below

To store files on a partition you need some structures indicating where your file is along with names, dates, permissions... These are called metadata and needs space on the volume so they're counted into used space of the drive that you see in the volume properties

Now drives work on blocks instead of bytes. You can't read a single byte without reading the whole block containing it. File systems may in turn use bigger blocks than the physical drive's block size to reduce metadata size required for bookkeeping and increase performance. If the file size isn't divisible than the file system's block size then the last segment of the file won't fill the whole block. The remaining data in the block will be wasted, making size on disk of the file bigger than the actual size. However size on disk can also be smaller than the file's real size

  • Sparse files consume space on disk only for their non-zero parts, so a 1GB sparse file with 1MB of non-zero data has size on disk of about ~1MB
  • Compressed files have size on disk equal to their sizes after compression
  • Resident files have size on disk equal to zero. Those files are small files that their data fit into the free space in the metadata part so they don't count into size on disk

In fact 3GB for the metadata is nothing compared to 1000GB. That is just ~0.3% of the drive. If your drive consists of lots of small files then the wasted size would be much larger. For example if the block size is 16KB then you'll waste 50% of the disk if you store files 8KB on average. That's why exFAT sometimes have so big size on disk. See Same files have different "size on disk" on new drive

ext4 also wastes more than NTFS with the default settings. They need ~1/64 (1.5625%) of the partition for metadata, and 5% for the reserved space. That's ~66GB on a 1TB disk. Now you see how small that 3GB is? See Is ext4 more expensive than ntfs?

See also Why does "Size on disk" vary when "Size" does not with the same set of files?

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .