In several applications, the Save and Load dialog windows are just not showing. And in notepad (which obviously handles the return code) is reporting an out of memory issue, which I'm sure is not the case; I think it's assuming it's out of memory as it can't get the window handle. In any case, there's something definitely wrong with Windows, but there's nothing in the event log. Any idea why this might happen?
I'm guessing Windows XP 32-bit with over 2 GB of RAM. There seems to be problems in Windows where it reports erroneously that it is out of memory, when in fact there is more than it needs. This is due to using a signed integer instead of an unsigned one to measure the amount of RAM (you know, those signed-unsigned comparisons the compilers warn us about).
You either have to remove RAM until you have 2 GB or less, or you need Physical Address Extension (PAE). Alternatively, if you are compiling a program, you may make it aware of more than 2 GB of RAM by using the /LARGEADDRESSAWARE linker flag for Microsoft's linker, but this is not the problem mentioned here. [source]
You may also want to try to update Windows XP to the latest version, which may contain updated binaries aware of larger addresses, which would solve the problem. There are also the /3GB, /4GB and /PAE options in boot.ini which will make the system aware that it is running more than 2 GB of RAM.
I hope this solves the problem. You may also try to switch to a 64-bit version of the operating system, but that is not always an option because of monetary costs of some operating systems and due to the availability of some drivers.
EDIT: I forgot to mention that you may also try to increase the size of the pagefile. It can help, especially if the pagefile was 2 GB and you increased the amount of RAM available without modifying the size of the pagefile.