Super User is a question and answer site for computer enthusiasts and power users. It only takes a minute to sign up.
Sign up to join this community
I have a lot of files that fluctuate between 1 and 50KB on a SCSI USB connected SSD. What would be the best allocation unit size to choose when formatting it to exFAT?
I am wondering what would be the fastest in reading / writing. I am considering 64K, although 32K seems better. I hope someone can help me figure out what is the smartest move to make!
We can use the ExFAT Specification to derive a formula that will tell us the best allocation size for a given usage.
Let's assume we are talking about a volume of size V, formatted with allocation size (cluster size) C and we intend to store files of average size S (all numbers are in bytes.)
All FAT filesystems, including ExFAT, store 2 copies of the FAT (file allocation table) each of which contains as many entries as there are clusters on the disk, with each entry taking up 4 bytes. This is the main source of Therefore the space dedicated (wasted) to the 2 copies of the FAT is approximately:
Additionally, each file will waste on average half a cluster, so we need to know how many files will fit on the disk. The number of files of average size S that will fit in the remaining space, once you allocate the 2 FAT copies, is approximately equal to:
Therefore, the total amount of wasted space or filesystem overhead is equal to:
The optimal cluster size corresponds to the minimum wasted space, therefore we compute the partial derivative of W with respect to C:
And solve the derivative for zero, keeping the positive answer:
Surprisingly (or not*) the optimal cluster size does not depend on the size of the volume, but only on the average size of the files.
If you can predict the average size of the files you will store on the device (just take an existing directory of similar data and divide its size by the number of files it contains), you can take the square root of the average file size and multiply by 4 to get the optimal cluster size for an ExFAT drive.
(*) Thinking about it, the result makes sense. If you double a volume while keeping the cluster size constant, you are doubling both the space allocated to the FAT and that allocated to files. Therefore the ratio of wasted to used space remains unchanged.
After some testing (thanks to a friend) I got these results:
After formatting it to exFAT with the allocation size set to:
So to conclude: 8192 (~ 8K) obviously wins when it comes to small files and Android project related files.
