I can do tethering with my cell phone.
Where I work, certain sites are blocked (gmail, yahoo mail, etc...). Is there a way to tell windows to go through the cell phone on certain sites, but for all others, to go through the regular network card?
If you don't use a proxy, you may need to do this with the windows routing table. By default the machine routes anything not on its local network to the internet through the preferred gateway. You can set up rules that force certain interactions to go through the network adaptor and its associated gateway set up when you tether your phone.
It's not trivial, but it's possible.
I would guess that your workplace will be less than pleased with you if they find out that you're tethering your phone, effectively bypassing any protections that have in place.
That said, you can do this through your browser proxy settings. They differ slightly by browser, but effectively, you can point specific addresses, or ranges of address, or names, to a different proxy. Create a local proxy that points to your tethered phone, and set up your browser to use that proxy for the addresses or names you specify.