Try taking a screenshot of some text with and without font smoothing and post a magnified crop here as an update to your question.
As I understand it, the way some Font-smoothing works is essentially by blurring the edges of the glyph. So it should be expected - to some extent.
There are several mechanisms for improving the appearance of fonts at low resolutions or low sizes. (e.g. Sub-Pixel Rendering) Not all of these are suited to modern LCD/LED displays.
Note that Sub-Pixel Rendering depends on the computer knowing the order and geometric arrangement of individual color pixels on the physical display. If this is wrong the results would be very poor.
from Wikimedia, photo by Peter Halasz. (User:Pengo)
Since some decades ago, displays have communicated their capabilities back to the computer using a Display Data Channel on one wire of the VGA, DVI or HDMI connector. I don't know if this includes pixel geometry or if the system could operate at all with this connection broken, but it might be worth testing.
Update: In the example OSX screenshot images, the main problem is JPEG artefacts, here is an enlarged section:
All that blotchy muddiness is the result of JPEG compression. To compare font smoothing you need a lossless format without artefacts - PNG is a good choice.
P.S. subpixel rendering won't show up in screenshots made using the computer alone. You need to use a digital camera with macro capability.
Resolution and Font size
To achieve maximum legibility with an LCD display you should always set the resolution to the native resulotion of the display. To increase the size of lettering used in windows, desktop-icons, system-dialogues and in applications you should use other means. For example http://hints.macworld.com/article.php?story=20080402181745803 or screen magnification