For Linux command sort, how do I force sort to load all input into memory and sort assuming I have enough memory? Or is it best to use a RAMDISK to store the input before feeding it to sort?
migrated from stackoverflow.com Jun 11 '11 at 19:49
This question came from our site for professional and enthusiast programmers.
You can specify the temporary directory to be nonexistant and change the main memory size parameter. This will however cause the sort to fail if you don't have enough mem:
Unit for the
Barring very non-standard filesystems the whole shebang will be read-cached any way (observe this simple in htop).
You can see the amount of buffering as well in
Observe how you can force the cache to be cleared by issueing
On the write side, the tmpfs feature in linux 2.4+ is perfect. It does the analogous of the read caching and you can manually limit it's size. This is my default /tmp mount:
I'll usually work on /tmp for longer periods of the day and use version control to push things into a (nonvolatile) repository.
So, shy from /write it yourself/ solutions, you should just use the kernel features that are there.
 I also symlink things like ~/.cache ~/.opera/cache etc. into /tmp/ Really lifts the burden of cleaning up, make things fly performance wise and keeps my SSDs in healthy condition
If whichever temporary directory "sort" uses supports "delayed allocation" - for example, ext4 or xfs, then it will do this anyway!
If the kernel decides there is enough memory, it won't need to write the data to disc; if the sort happens sufficiently quickly that the kernel doesn't write the data to disc, then the files are deleted, they will be thrown away and forgotten about without any writes at all.