Originally, there were two versions of VirtualBox, the non-free one you are using (which is probably earlier than version 4.0.x) and VirtualBox OSE, consisting only of free/open source software. The non-free version adds RDP and USB 2.0 support, among other things. In order to use VNC instead of RDP with this earlier version, you must install VirtualBox OSE instead. It is available in the
virtualbox-ose package in the Ubuntu universe repository.
VirtualBox version 4 and later is only available in an open source version, with an add-on pack to enable RDP and USB 2.0 support. If you'd like to install this version, you can obtain it via Oracle's official APT repository. It does not include the non-free add-on pack with RDP, which must be downloaded from Oracle separately.
Just run the following commands to add the repository and install the latest version of VirtualBox 4.0:
sudo apt-add-repository "deb http://download.virtualbox.org/virtualbox/debian natty contrib non-free"
wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add -
sudo apt-get update
sudo apt-get install virtualbox-4.0
This assumes you're running Linux Mint 11 "Katya". For version 10 "Julia", replace
maverick. For version 9 "Isadora", replace it with