Vista allocates memory for faster performance and can release any of this cached memory if another program needs it, and is very efficient at managing memory, so I don't think this is the issue.
I suspect malware. After you follow the steps below, do an IE reset using Internet Options Advanced Tab, then clean out your IE temporary Internet files and user temp files folder.
Follow the order given below to disinfect your PC.
Make a boot AV disc then boot from the disc and scan the hard drive, remove any infections it finds, I prefer the Kaspersky disc myself. The New 2010 Kaspersky disc can update the AV dat files if you are connected to the internet at the time of scan and is suggested to update before the scan.
Install free MBAM, run the program and go to the Update tab and update it, then go to the Scanner Tab and do a quick scan, select and remove anything it finds.
When MBAM is done install SAS free version, run a quick scan, remove what it automatically selects.
These last 2 are not AV softwares like Norton, AVG etc they are on demand scanners that only scan for nasties when you run the program and will not interfere with your installed AV, these can be run once a day or week to ensure you are not infected. Be sure you update them before each daily-weekly scan.