The answer is "it depends"
Some 32-bit 'applications' will not work in 64-bit mode, for example graphics drivers and shell extensions (such as Tortoise SVN). In such a case you need a 64-bit version.
For other normal applications, then it depends on what they do.
If you happen to have, for example, 16GB of memory and are running SQL server with an extremely large database, then having 64-bit version of SQL is very important.
If, however, you have 4Gb of memory (with almost 1gb 'stolen' by the video card), then you may be running low of memory and using 64-bit applications (which may require more memory) may actually slow your system down.
Even ignoring memory considerations, you can not say for certain that a 32-bit application running in 64-bit mode will be faster or slower than the 64-bit equivalent, nor whether the application will be faster or slower than when running on a 32-bit operating system.
Personally, if I'm looking for a utility or small application for my machine, I will check that it works with 64-bit operating system, but whether the application is 32-bit or 64-bit is not normally high on my priority list.
According to Process Explorer, I currently have 25 64-bit processes and 28 32-bit processes running, ignoring Chrome)