Why was the %Temp% folder invented?
Why will any program ever need to have to have a temporary folder for data? Can they not just use the RAM or any other temporary storage?
The usage of "temporary" files goes back to the times when computers ran MS-DOS and had 640 kB of physical memory and no paging. Playing memory tetris with device drivers and TSRs, hopelessly trying to free up some RAM, was a common (though not exactly "favourite") past-time for PC owners.
These days, %TEMP% is still useful as a place to keep files that don't need to be permanently saved. For example, when you download a document from a website and choose "Open" instead of "Save", it still has to be saved to a file locally before being given to the program – this is the only 100%-supported method of passing data to other programs. (After all, if it was presented as a file on the website, it's simplest to save it as a file again.)
Similarly, if you open a file directly from within an archive, it is unpacked to %TEMP% first – the archiver program takes care of decompression so that other programs won't need to. Often, you can even edit the file in the program, and it will be repacked afterwards. (In comparison, if the program had to unpack the file itself, then it would have to repack it every time you save, even if that is very rarely necessary.)
Programmers and power users have been using locations called
The biggest example of this is installers. Many installers are single files and when you see that bar that takes ages on large installations, it is usually expanding the files to the %temp% folder.
Quite often I write batch scripts that require a temporary/staging folder. They are only tiny, and I usually always end up using
As for why not use the memory, the still holds true today - it isn't large enough for everything. My first laptop had either 2 MB or 4 MB of memory (I forget), and while many programs were tiny, there were often a few that were in the 5-10 MB range (MS Office used to be about 20-30 MB for the whole suite!).
Now, the average is a lot higher (entry level machine is about 2-4 GB, and power users are 8-16 GB, power users with a bit of money 16 GB+), with a few exceptions such as scripts, the memory could really be used a lot more than it is today.
However, until the entry level machines are in the 16-32 GB Range (give it a few years!) I don't think we are going to have many advances.
With software I've written, I have used temporary folders for processing aircraft CVR files. The files are generally 512 MB in size. During processing the files go through a range of formats.
I start by copying the original file to a Temp folder where all processing takes place.
I use one library to convert from format A to format C (automatically via format B). This library only takes an input file name. It decides on an output file name in the same folder. It's all out of my control.
I use a second library to convert from format C to format D.
The final directory the files are placed in is visible to the user, so I don't want them to see the files going through their transitions, or for them to see partial files, only the final output. One all processing is complete, the required files are moved to their final destination and the