When you install msysgit on windows, you get an integration option which will allow you to use it from the "Run" menu.
I've never installed it with that option, but it seems like that might cause it to make a fuss whenever it can't find the git binaries in your PATH.
I think, and don't quote me on this, that there's some weird hacking to make git transparently available through the windows command prompt, but you might try looking for installed git (msysgit?) and uninstalling it? Or if you want to add it to your path, in Control Panel > Advanced > Environment Variables, it'll mightlook something like:
GIT C:\path\to\git.exe
problem uninstalling GIT on windows gives instructions if uninstalling it isn't working.