Yes, you can have only Windows installed, but there are some other things to be aware of:
To have full hardware support, you will need a copy of OS X Leopard or preferably Snow Leopard since Snow Leopard has a newer version of the Boot Camp drivers and device software. When you are booted into Windows and insert the OS X disc, you will be prompted to install the Boot Camp drivers and utilities. These Boot Camp drivers are used for things like the Apple keyboards, trackpads, mice, special function keys for brightness, etc. to function in Windows. You can download a version of the Boot Camp drivers from Apple's website, but it is only an update and doesn't include the full suite of drivers and device software. Since the full Boot Camp software is on the OS X disc, you will need a copy of the OS X on-hand, but you don't have to have to keep OS X installed.
If you are installing Windows on an older Mac, you might need a temporary installation of OS X to update the EFI firmware to allow BIOS emulation which is needed to boot Windows. If you do need to update the firmware, you can wipe off OS X after it is updated. Some newer versions of Windows such as Vista SP1 and 7 have a EFI boot mode, but Microsoft's EFI boot code is not compatible with Apple's EFI. Because of this Windows limitation, the BIOS emulation is necessary to boot Windows, however this BIOS compatibility is in the computer's firmware and does not require an OS X installation in order to function. Also, you can safely format the hard drive as MBR-only if you want only Windows (and/or Linux) installed. If you are also running OS X, you would want to be using GPT or a GPT/MBR hybrid.