I realize that this question was asked two years ago, but I don't think any of the answers here are complete, so I'm going to give it a shot anyway.
To answer the subject line, USB's limit is the limit of its mass-storage class (MSC) spec. This spec is in turn based on the SCSI transparent command set. (See also http://en.wikipedia.org/wiki/USB_mass_storage_device_class#Device_access). Looking at the SCSI read commands (http://en.wikipedia.org/wiki/SCSI_Read_Commands#Read_.286.29) you can see that there are several versions of the API. The oldest supports 21-bit block addresses (and a 1GB limit), a later version supports 32-bit LBAs (imposing the 2TB limit many cases have today), and the latest has a 64-bit LBA which will impose an 8ZB limit (that's 8 giga-terabytes!).
Modern USB cases will support 64-bit LBAs and should support any drive you're likely to find. Older cases won't support this and will misbehave if they are attached to a drive larger than 2TB.
Assuming your USB case supports the drive, then there's the issue of partitioning the drive. The MBR partition scheme (used in various forms since MS-DOS) uses a 32-bit block index. So it has a limit of 2TB. The solution used today is to switch to the GUID partition table (GPT) format which uses 64-bit block numbers.
All modern operating systems (including Windows, Mac OS X and Linux) support GUID partition tables and should be able to support a >2TB drive as a data disc.
If you're a Mac or Linux user, then you should be able to boot from the drive as well. Macs have EFI firmware and support booting GPT volumes. Modern Linux systems have code in the GRUB bootloader that, I'm told, will allow booting Linux from a GUID drive without EFI firmware..
If you're running Windows, however, you may or may not be able to boot from the drive. You will need EFI firmware on your motherboard, and you will need to be running a 64-bit version of Windows. Microsoft requires both in order to boot from a GPT drive of any size. This means that if you are running 32-bit Windows or have BIOS-only firmware (without EFI), you will want to make sure your boot disk is 2TB or smaller, so you can apply an MBR partition.