If I have 2 computers with wireless adapters (1 has 802.11g, 1 has 802.11b) and I have a wireless router 802.11g. Can I setup a home network (i.e. have 2 computers talk to each other)? Do I need to have internet access from an ISP (e.g. comcast) in order to achieve that?
You don't need internet access.
Just set up the router (defaults should be fine, but make sure you enable encryption and set a password on the router settings), then configure each PC to receive an IP address via DHCP (this should also be default). After that, the computers should be reachable from each other (of course, further setup may be required, but without knowing your OS or software configuration, it would be impossible to elaborate further).