18

When moving a file within one drive, the file is not copied-and-deleted. The table that refers to the files is just updated. And as far as I know, that is not the case on 2 drives on an HDD. But SSDs are different, there is no physical space dedicated to each drive. (source)

So my question is what happens when a file is moved from one drive to another on the same SSD, are the bytes copied and the original deleted, or is some table updated, thereby thrashing the SSD less?

There's already a duplicate question here. But both answers claim:

each partition will have it's own physical area of the drive to itself

and

Partitioning a hard drive actually designates physical regions for each partition. [and in a comment:] SSD is still a hard drive, it just doesn't have a disk.

As far as I know that's wrong. See here.

So will someone who knows more about SSDs please tell me if they are correct in their assessment despite their mistake?

  • 14
    The accepted answer there is right: each partition has it's own, independent filesystem. Each filesystem decides for itself how it utilizes blocks assigned to it. To do what you propose the firmware of SSD, filesystem(s) and userland tools like Linux mv would have to cooperate, mixing abstraction layers greatly. – Kamil Maciorowski Dec 26 '18 at 19:23
  • 2
    @Kamil: If the OS implemented it, mv would actually need to do less than it currently does, I suspect. (That is, the OS would just need to make sure that a cross-filesystem rename() succeeds instead of failing as currently happens.) – user1686 Dec 26 '18 at 20:04
  • 16
    "2 drives on [an SSD/HDD]" -- I think you mean to say 2 filesystems or 2 partitions on one SSD/HDD. Remember that the last "D" in both acronyms is "Drive", so you're saying 2 drives in 1 drive, which doesn't make sense. – JoL Dec 26 '18 at 20:34
  • 1
    Take for example the Disk Management dialog. It says Change Drive Letter and Paths when referring to a partition/volume. – ispiro Dec 26 '18 at 20:51
  • 4
    @ispiro Is that on Windows? Seems like an awful way to confuse users. I can only guess that they came up with the term "Drive Letter" before drive partitions where implemented, and then they ended up representing drive partitions as standalone "drives". So now you can have multiple Windows "drives" representing partitions of one hardware drive... – JoL Dec 26 '18 at 21:15
38

As far as I know that's wrong

The quoted description is half-correct, half-wrong. But it's also half-wrong for HDDs too.

Partitioning a drive designates logical regions for each partition. The OS doesn't care about physical locations at all – it just asks the drive to "read logical block #31415926" and the drive itself decides where the data is located. This works the same way for magnetic and flash memory.

It's actually the same as with HDDs from the last 20–25 years: although early operating systems used physical cylinder/head/sector locations, that's long gone now. You don't know precisely where on which platter LBA #1234 is kept. HDDs even remap bad physical sectors automatically, so the same LBA can suddenly be read from a completely different physical area – just like with SSDs.

So with both HDDs and SSDs, the OS just has a range of LBAs (e.g. 0–999999) to read and write data from. The purpose of partitioning is to allocate sub-ranges in it – e.g. partition A gets 10–499999, partition B gets 500000–999999. Each partition has an independent filesystem, and filesystems inside each partition cannot reference data outside it – they cannot cross the partition boundaries. (For example, partition A cannot have a file whose data is kept in sector #600000.)

As a result, all files moving from one to the other have to be copied in full.

(That said, in theory the OS may be able to ask the disk itself to duplicate data from one area to another (e.g. "copy LBA #1234 to #567890"), without having to copy it to the main memory and then back, and of course this would completely bypass partition boundaries. This could make use of the SSD's "flash translation layer", for example. But in practice, as far as I know, this isn't done.)

| improve this answer | |
  • I assume you're right. However, notice that there is another option. The drive could decide that sector #001 on drive#1 would now switch with sector #123 in drive#2 (i.e. sector#001 on drive#1 will now refer to the physical data that used to be referred to as sector #123 in drive#2) thereby moving the file without having to copy the bytes. So moving a TB of data can in principle be near-instantaneous. – ispiro Dec 26 '18 at 19:53
  • 15
    The drive doesn't know about files or filesystems, and therefore cannot make such decisions by itself. It needs to receive a request from the OS for this to happen. As I already mentioned in the last paragraph, it's certainly technically possible but operating systems usually don't bother with this for cross-filesystem copies, and I doubt they do for same-filesystem copies either (more commonly it's done at FS level). – user1686 Dec 26 '18 at 19:59
  • 6
    Certain SSDs do implement block level de-duplication. E.g. Sandforce (en.wikipedia.org/wiki/SandForce#Technology) were one of the first to implement this and their controllers made their way into multiple SSD manufacturer's products. Sandforce controllers had a "write amplification" factor of less than one -- which means they wrote less data to flash storage than what the OS sent to the drive. As a comparison, SSDs generally have a write amp factor of greater than one. – hojusaram Dec 27 '18 at 3:31
  • 2
    @hojusaram: True, but it's still post-factum deduplication – reducing flash writes, but the data is still read, copied from disk to OS memory and then back to disk controller. What ispiro meant I think is the SSD equivalent of "reflink" or "copy on write" (e.g. cp --reflink) that the OS could explicitly ask the disk to perform on its own. – user1686 Dec 27 '18 at 7:37
  • 1
    It would be interesting to discover how APFS deals with this, as the partition boundaries are no longer fixed, they're mutable without intervention from the user. – Tetsujin Dec 27 '18 at 9:41
9

What happens when data is written to a Solid State Disk is worthy of several articles (good summary here), because it is very complicated and depends on the underlying technology. The short story is that SSDs in general cannot write zero bits to memory. Instead, they have to zero out (erase) a whole section of memory, and then they can store data after that by just writing the ones to it. Typically these days they write blocks of 512 bytes but erase a page of 8 blocks which is 4096. This, and the fact that each write/erase cycle causes some physical wear of the memory and the memory does eventually wear out, make SSDs very different than spinning magnetic HDDs.

Setting that aside, SATA drives (and AFAIK SAS drives) do not implement a native command to copy data from one sector to another. (Or at least nothing in the SATA or SAS spec requires them to, so the OS cannot count on such a command being available.) So a file copy across a partition will involve reading the data from one drive sector into host memory and then writing it back out to the drive at a different sector.

This is because as far as the OS is concerned, a drive is a set of numbered logical sectors, and all it can do is read from sectors and write to sectors. The OS cannot tell the drive to remap sectors.

Furthermore, the file system (HFS+, NTFS, ext3, etc.) is a set of data structures that impose order on a set of logical blocks. Those data structures implement "files", "file names", "directories", "permissions", etc. So, yes, when you move a file from one directory to another, it is not copied; only the file system data indicating which directory the file is in gets updated.

The concept of a partition is that it is a set of logical sectors on the drive claimed by a single file system. The corollary to that is that a file system may not access sectors outside of its partition. In large part this is a safety feature, but it also flows from the fact that the file system's data structures are all built around accounting for every sector of the drive under the file system's ownership, and it is non-trivial to add or remove sectors to those structures. This is why you have to run special routines to adjust the size of a partition and also why file systems insist on running on a contiguous set of sectors.

So it is impractical and dangerous to implement a file copy as just transferring sectors from one file system to another. On a spinning magnetic drive, it would also be a performance nightmare, because although the drive will make exceptions for bad sectors, in general it arranges for sectors to be physically located in such a way as to optimize the read and write speed of consecutively numbered sectors.

Additionally, 2 file systems may not store file data the same way on disk, meaning swapping sectors would not work even if it were practical. Even if they are the exact same file system types, say NTFS, one may be using encryption or compression and the other not, or both may encrypting the data, but with different keys. It is not a requirement that the data in the file is exactly what is stored on the disk, all that has to be stored is a reversible transformation of the data, so that the file system can get the data of the file by doing something with the data on disk. So unless both file systems are using exactly the same transformation, simply swapping sectors would not accomplish the goal of transferring the file data.

For all these reasons, it is just too much work for too little gain for the OS writers and file system writers to implement a feature that optimizes moves across partitions for SSDs. So any cross-partition move will be a read and a write.

Inside the SSD, it is a slightly different story. Although the OS did not tell the drive it is copying data from one place to another, writes to SSDs are so expensive (and complicated) that SSD controllers do a lot of work to minimize writes. Some SSDs go so far as to try to detect when a sector being written to storage matches a sector already stored and mark that physical piece of memory as now mapping to 2 different logical sectors rather than copying it, doing at the internal drive level what the OS could not.

But don't count on it.

| improve this answer | |
  • 1
    Doesn't your last paragraph imply that the filesystem must be same? I would assume the SSD doesn't know which filesystem is running on top. If for example one partition uses compression and the other doesn't, a copy by the SSD would possibly corrupt the file. – blablabla Dec 27 '18 at 16:25
  • @blablabla The last paragraph assumed both file systems store the actual file contents on disk without alteration. I made that explicit now. – Old Pro Dec 27 '18 at 21:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.