Does anyone know of a utility similar to SpeedGuide.net's TCP Optimizer or some other way to accomplish the same thing on Mac OS X (preferably with Snow Leopard)?
You can always check out any Linux based howto on this topic, Mac is also only a Unix system. Most guides you'll find online are targeted towards servers (although Mac based servers are really rare). The following was taken from ESnet:
For a more in-depth discussion of the topic check the following lengthy post Performance Tuning the Network Stack on Mac OS X.