0

On a Linux FS such as ext4, would temporary files such as logs and other /var stuff tend to end up on the same physical blocks over time? For example, with log rotation, would new logs tend to end up on the same blocks where previous (now deleted) log or other temporary files used to be?

(The focus of this question is primarily Linux/ext4, but insights re other filesystems and OSes are also welcome.)

Ignore anything that's mounted to tmpfs or to its own partition. I'm talking only about the stuff directly in the root partition. And assume that /var as a whole is not on a separate partition.

I'm only mention /var because I'm assuming the main thing that accumulates changes over time is the various stuff in /tmp, /var, and /run. The rest is mostly installed packages and configuration, which change much less often. Of those 3, /tmp and /run are often mounted to tmpfs, so we can ignore those. That leaves only /var. But the path really doesn't matter. The question is about allocation behavior of short-lived files anywhere (and also longer-lived ones that are modified often).

IOW, how much do such files move around on the disk?

Context: (can ignore)

The context is an LVM2 snapshot of a root filesystem. I'm assuming the main thing that accumulates changes over time is the various stuff in /tmp, /var, and /run, and that most of that are temporary files of various lifetimes. Then:

  1. If these files tend to end up on the same limited set of blocks, the snapshot will only need to back up those relatively few blocks.
  2. OTOH, if these files end up on different blocks every time, then eventually many of the unused blocks will get written to (with other blocks becoming unused in their place). In this case, the snapshot will have to back up a much larger set of blocks, and will grow to a larger size.

I'm aware of other questions regarding selecting an LVM snapshot size, size recommendations, etc. But I'm more interested in the more general file allocation angle. So you may ignore the LVM context.

Improve this question
5
  • short answer, no. A file that exists for a long period of time will remain in the same blocks, but will allocate additional blocks (as near to the existing ones as possible) as needed. new files however will not be grouped with the existing files in a directory, or anything like that. about the only way you could isolate physical storage by path would be to mount /var, etc from their own partition/filesystem. the logical organizational structure of the directories and files has no relation to the physical storage location on the media.
    – Frank Thomas
    Commented Feb 9, 2021 at 20:57
  • 1
    You need to check mount points to verify the underlying filesystems in use. /var/run is likely to be a symbolic link to /run, which is likely to be the mount point for a tmpfs (and not ext4). tmpfs utilizes RAM and swap space, so your question really does not make sense, i.e. it's based on a faulty premise.
    – sawdust
    Commented Feb 10, 2021 at 0:02
  • @FrankThomas What I was thinking was, as short-lived files get deleted, new files can reuse those same blocks where the previous files were, give or take fragmentation effects. So if we're looking at time scales longer than the average lifetime of these files, they should tend toward the same physical locations. If there's an allocator preference towards a certain area of the disk (e.g. to minimize seek times), that effect should be stronger still. Something along those lines.
    – MichaelK
    Commented Feb 10, 2021 at 5:29
  • @sawdust /var/run is only one subfolder. There are other things in /var that aren't mounted to tmpfs. Assuming /var itself isn't a separate partition, some of these will be in the root partition directly. I've edited to clarify.
    – MichaelK
    Commented Feb 10, 2021 at 5:37
  • @FrankThomas Also, forgot to mention: I've read that on Linux there is a practice to write changes to a temp file and then rename, to make changes more atomic. This is probably less relevant to logs, but for some other files this should significantly shorten their actual lifetime, compared to the apparent lifetime of the corresponding filename.
    – MichaelK
    Commented Feb 10, 2021 at 5:47

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .