Virtualbox has a single installer for windows that installs 32 bit application on 32 bit os and 64 bit application on 64 bit os. Is it any way to install 32 bit application on 64 bit windows Vista? I need this for COM automation, it's different for 32 bit and 64 bit.
The last VirtualBox release that didn't have the 32-bits and 64-bits versions bundled together was 2.1.
The 32-bits installer for version 2.1 can be downloaded from here.
If VirtualBox version 2.1 answers your need, then your problem is solved.
Otherwise, here are two ideas.
Idea #1: Install the old 32-bits VirtualBox version 2.1, then try to upgrade it to the latest.
Maybe the latest installer will prefer to upgrade the installed 32-bits version.
Idea #2: Download the VirtualBox sources and compile the 32-bits version (the Open Source Edition does not include an installer). That's maybe too much work, though
The actual virtual machine monitor in virtualbox is a device driver in windows (exactly the same as VMware or VirtualPC). The Networking and USB components of virtualbox are also device drivers.
A 64bit os must have 64bit device drivers, so you cannot install x86 virtualbox on windows x64.
Is this COM automation of VirtualBox itself or of an app running in a VM? You can run a 32-bit OS on a VM running on 64-bit VirtualBox.
Have you tried this yet? Most 32 bit applications should run just find under Vista 64. If you run into any problems or if you have any specific problems errors, please update your question with more details.
Because of the low-level hooks that VirtualBox (or any VM software) makes into the host OS and its hypervisor, I don't think its possible to install the 32-bit version of the host software onto a 64-bit operating system.
Unfortunately, I can't find any documentation for VirtualBox's installer's command line switches, which might let you override the OS's bit-level detection. My guess is that there is just no way, for the reason I described above.
If all else fails...
The easiest way would to be to install Vista 32 bit as a guest OS. If you have a retail version, you should be able to use the same key for that without problem. Then you could program on that guest OS and install Virtualbox 32 bit on that guest OS. This first layer will be hardware accelerated an the second layer will then be software accelerated, now it's only dependent on the amount of memory you have available for this.
If you don't have the amount of memory needed or you need the second layer to be hardware accelerated then you would need to do a dual boot.