Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If I have a text file in Unix that contains N-many independent entries (e.g. records about employees, where each employee has a separate record), is it expected that this file will take up less space than if I split the file into N files, each containing the entry for one employee? in other words, can one save significant space on unix file systems by concatenating many files together, or is the difference negligible? thanks.

share|improve this question

It depends on the relationship between the record lengths and filesystem block size.

For example, if the record lengths (modulo the block size) are randomly distributed, concatenating the data into fewer files will reduce wasted space. The last block in each file will be 1/2 full on average. This is what's called internal fragmentation. If you cut the number of files in half by combining files, you would reduce the number of 1/2 full blocks in half.

Otoh, if every record is always a multiple of the block size, every file will always completely fill every block allocated to it (zero internal fragmentation) no matter how many records it contains. In that case, the only savings from concatenating files would be whatever came from having fewer directory entries.

share|improve this answer
On the other other hand, if the records are short, each one will get a full disk block, wasting much space. – vonbrand Jan 23 '13 at 19:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .