0

Suppose you have a file called foo.txt that is 500 bytes in size. Suppose you create a new file that is 10 bytes in size and save it as foo.txt, thus overwriting the file. Does the OS (Windows and Linux) automatically ensure that the space no longer used (410 bytes) is freed?

In a second scenario, imagine you used the dd utility to create both the old and new foo.txt as described above. Will the OS automatically ensure that the space no longer used is freed?

I imagine that behind the scenes, all writes to a file use the same OS system calls and thus handling of overwrites will be consistent across programs...

Improve this question
3
  • 1
    500 bytes and 10 bytes are below available logical sector sizes of HDD/SSD and below common block sizes of various filesystems. Did you deliberately choose such low values? In general the answer would be different for 500k/10k.
    – Kamil Maciorowski
    Aug 14, 2019 at 4:03
  • There are many filesystems, each may have its own quirks. To get to them in an answer I would first need to repeat what other general answers say, and then it would get broad. Just to signalize: data inside metadata structures; blocks shared among many files (COW); delayed maintenance (space is freed later); compression. And finally dd vs mv: you can write to the same file or overwrite it with a new file; in the latter case the old file may still be intact and in use.
    – Kamil Maciorowski
    Aug 14, 2019 at 9:42
  • @KamilMaciorowski The numbers were arbitrary. The goal was to show that the newer file was smaller in size than the older one.
    – Jet Blue
    Aug 14, 2019 at 19:29

2 Answers 2

Reset to default
2

It sounds like your question stems from viewing the process as involving several things: all unused space being available for other use, and writing a file the same name as an existing file is being written to the same location. Both of these premises are inaccurate. Also, you talk about the space no longer used being freed. The way it works, those terms are kind of synonymous; space is either allocated to a file or it isn't.

As davidgo described, drives work in whole sectors or blocks. I'll just refer to the space allocation units as "blocks" to keep it simple. Space is allocated in whole block units. A 1-byte file is allocated the entire block, so if you're discussing tiny files, anything under a block in size is still assigned an entire block. Small (sub-block) files have unused space in the block that is not accessible for another purpose. You could talk about a large file that uses multiple blocks being replaced by a smaller file that uses fewer blocks. In that case, there are whole blocks that are no longer needed.

The old file isn't really overwritten. The new file gets saved in another location, using as many blocks as it needs. The reference in the filesystem's file table to the old file's blocks gets modified. Those blocks become unassigned to any file and available for reuse. The old file's content doesn't get deleted as part of this process, it is just ignored until the space is needed. That's why you're able to recover deleted files.

You asked whether this would be different if using dd. dd can be used in a lot of ways. If you limit the discussion to simply writing a new file using the same name, that would work the same way.

Improve this answer
2
  • Ah, so the old and new file's data do not reside in the same location on disk? That is, the OS does not try to be clever and reuse the space that the old file used. But instead it allocates separate space for the new file, and deallocates whatever space the old used?
    – Jet Blue
    Aug 14, 2019 at 19:35
  • @JetBlue, right. There's really no connection between the old and new files other than file table "bookkeeping" and bundling the two tasks into one command.
    – fixer1234
    Aug 14, 2019 at 20:09
1

As @KamilMaciorowski said, a key component of the question is the block size in disk. It would be vanishingly rare to find a modern filesystem with a block size of less then 512 bytes (and most have much larger block sizes). This is relevant because it means that if you deal in file operations smaller then the block size, those operations use as much resource as the block size - so the answer to your question is NO - the 400 byte file, which takes 1 block which will be replaced with another file using 1 block. The original block will likely be freed (but might be overwritten) but you have not saved any disk space

I suspect you know this but for others - It's also worth noting that when a block is freed it is not usually overwritten/blanked out and can often be recovered. Thus is doubly true on SSDs which further abstract what us in disk from what the OS sees (thanks to wear levelling).

Improve this answer
4
  • The numbers were arbitrary. The goal was to show that the newer file was smaller in size than the older one
    – Jet Blue
    Aug 14, 2019 at 19:31
  • Thanks for the insight on block size! What I was curious about is whether the OS "frees" the old file, before "overwriting" it with the new one. Or does a user need to explicitly delete the old file first, before creating a new one with the same name, if they want to avoid "leaking" space in the file system.
    – Jet Blue
    Aug 14, 2019 at 19:40
  • This is typically handled automatically by the OS and an attempt to rewrite a file - if successful will delete the old one and create a new one. You can't (generally soeaking, without a lot of effort to get round the OS) create multiple files with the same name at the same location at the same time. This limit is typically also imposed by the filesystem as well as OS.
    – davidgo
    Aug 14, 2019 at 19:50
  • "will delete the old one and create a new one" - thank you! This is what I wanted to confirm.
    – Jet Blue
    Aug 14, 2019 at 21:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .