I'm looking for a CVS client which works from the console. All windows clients that I've found are GUI's.
In Cygwin there is command line CVS available. It's pretty large package just for CVS client, but you'll get other useful things too, for example standard unix command line tools like
You can also run Cygwin tools from normal Windows command prompt (