You can use FreeNX on the VPS and NX Client on the Windows/Linux/Mapple PC that you have at home/work.
The great thing about the FreeNX solution is that you can pick up your desktop from any machine without having to shutdown and restart your session.
Depending on how your VPS is setup you may want to check the session is not using too many resources. Personally I run Gnome without too many desktop effects or the browser (I can run that on my own client machine) and I haven't had any problems.
Google 'FreeNX headless' for your chosen distro, sometimes you need to install a virtual frame buffer (XVFRB) here is what you need for Ubuntu FreeNX:
https://help.ubuntu.com/community/FreeNX
Enjoy!