One reason is that the security model on OS X is quite strong: even administrative users have to enter their password before they're allowed to modify system files.
Windows XP didn't have anything like this: Admins could do everything they wanted by default, but non-admins couldn't do much at all, pretty much requiring everyone to run as Admins all the time and leave the door wide open. Lately they've been moving towards a more Mac-like model, but in Vista this resulted in "Do you want to allow this behavior?" boxes popping up so often that people got into the habit of ignoring it, or worse, turning it off entirely.
Mac users usually only have to confirm behavior when installing software, so if they're asked to enter a password at any other time it immediately arouses suspicion.
Another reason viruses and worms are so rare on OS X might be because much of the core of OS X is open source, in particular based on FreeBSD, an old, mature code base known for its security.
Most of the malware I've heard about for the Mac has been classic trojans: just trick the user into thinking they're downloading and installing something they want. No OS in the world could ever protect against that.