When should I use /dev/shm/ and when should I use /tmp? Can I always rely on them both being there on Unices?
Since RAM is significantly faster than disk storage, you can use
To answer your questions: No, you cannot always rely on
Okay, here's the reality.
Both tmpfs and a normal filesystem are a memory cache over disk.
The tmpfs uses memory and swapspace as it's backing store a filesystem uses a specific area of disk, neither is limited in the size the filesystem can be, it is quite possible to have a 200GB tmpfs on a machine with less than a GB of ram if you have enough swapspace.
The difference is in when data is written to the disk. For a tmpfs the data is written ONLY when memory gets too full or the data unlikely to be used soon. OTOH most normal Linux filesystems are designed to always have a more or less consistent set of data on the disk so if the user pulls the plug they don't lose everything.
Personally, I'm used to having operating systems that don't crash and UPS systems (eg: laptop batteries) so I think the ext2/3 filesystems are too paranoid with their 5-10 second checkpoint interval. The ext4 filesystem is better with a 10 minute checkpoint, except it treats user data as second class and doesn't protect it. (ext3 is the same but you don't notice it because of the 5 second checkpoint)
This frequent checkpointing means that unnecessary data is being continually written to disk, even for /tmp.
So the result is you need to create swap space as big as you need your /tmp to be (even if you have to create a swapfile) and use that space to mount a tmpfs of the required size onto /tmp.
NEVER use /dev/shm.
Unless, you're using it for very small (probably mmap'd) IPC files and you are sure that it exists (it's not a standard) and the machine has more than enough memory + swap available.
In descending order of
Since you are asking about a Linux specific tmpfs mountpoint versus a portably defined directory that may be tmpfs (depending on your sysadmin and what's default for your distro), your question has two aspects, which other answers have emphasized differently:
Conservative edition (mixture of conventions from FHS and common use):
Where tmpfs excels
Wanna hear a ridiculous story? My first
Where tmpfs is appropriate
The appropriate use of tmpfs is to avoid unnecessary writing of volatile data. Effectively disabling writeback, like setting
This has very little to do with performance, and failing this is a much smaller concern than abusing fsync: The writeback timeout determines how lazily the disk content is updated after the pagecache content, and the default of 5 seconds is a long time for a computer – an application can overwrite a file as frequently as it wants, in pagecache, but the content on disk is only updated about once every 5 seconds. Unless the application forces it through with fsync, that is. Think about how many times an application can output a small file in this time, and you see why fsyncing every single one would be a much bigger problem.
What tmpfs can not help you with
Where tmpfs sux
Keeping cold data. You might be tempted to think that serving files out of swap is just as efficient as a normal filesystem, but there are a couple of reasons why it isn't:
Use /tmp/ for temporary files. Use /dev/shm/ when you want shared memory (ie, interprocess communication through files).
You can rely on /tmp/ being there, but /dev/shm/ is a relatively recent Linux only thing.
/dev/shm is used for shared virtual memory system specific device drivers and programs.
If you are creating a program that requires a virtual memory heap that should be mapped to virtual memory. This goes double so if you need multiple processes or threads to be able to safely access that memory.
The fact is that just because the driver uses a special version of tmpfs for it, doesn't mean you should use it as a generic tmpfs partition. Instead, you should just create another tmpfs partition if you want one for your temporary directory.
One more stab at the back of /dev/shm: As of 2014, its permissions got stricter, you need to go through loops to write to it if you are not root:
In PERL, having 8GB minimum on any machine (all running Linux Mint), I am of what I think is a good habit of doing DB_File-based (data structure in a file) complex algorithms with millions of reads and writes using /dev/shm
In other languages, not having gigether everywhere, to avoid the starts and stops in network transfer (working locally on a file that is located on a server in a client-server atmosphere), using a batch file of some type, I will copy the whole (300-900MB) file at once to /dev/shm, run the program with output to /dev/shm, write the results back to the server, and delete from /dev/shm
Naturally, if I had less RAM, I would not be doing this. Ordinarily, the in-memory file system of /dev/shm reads as a size being one half of your available RAM. However, ordinary use of RAM is constant. So you really couldn't do this on a device with 2GB or less. To turn paraphrase to hyperbole, there is often things in RAM that even the system doesn't report well.