I ended up buying a computer that came preloaded with Windows 7 Pro 64-bit French version. Since it's not an Enterprise/Ultimate edition - I don't have the option to install/uninstall languages for it.
I've found a solution on this website that pointed me to https://sites.google.com/site/secrethowtos/
I followed the guide to the tee and after I rebooted the system(the last step) my cursor showed up on a blank, black screen and nothing else happened. I can move the cursor, but nothing else seems to work. I tried to reboot again, tried to run Last Good Known Configuration and tried to do Startup Repair, but ended up having to Restore to original state with French language.
Why I couldn't boot into the system after I installed the new language and removed the old one.