Some BIOS setups have a separate selection for "drive order" and will only boot from hard-drives in this order where hard-drive is selected as a boot option (so if the first drive has a boot loader installed, none of the others get a look in). My main desktop machine seems to behave this way (USB drive is a separate option, though it only boots of the first USB drive present if there is more than one, as is CDROM but in this case it only boots from the first PATA or SATA drive it sees with boot media, and so on).
If your BIOS is behaving in this way you will need to install your boot loader (lilo, grub, the Windows loader, or what-ever else) on one specific drive, configure it such that it can boot all the OS options you have installed, set that drive in your BIOS as the first drive, and boot from that letting the boot loader give/make the further choice as to which OS to to go with.
If you have a BIOS that functions this when then the other option in this instance is to have USB as your first boot option, set USB above "hard-drive" in the BIOS boot order, and install
grub to the USB device during the Ubuntu install (this is controlled in the "advanced" set of options in the last stage of the installation options). That way grub will be given control if said USB device is connected at boot (so Ubuntu will load) and if not the USB phase will pass and the first fixed harddrive with a boot loader will be booted from (presumably loading Windows).
Edit: some BIOSes have a "hit-a-certain-key-for-boot-choice" option presented for a few seconds at startup. My netbook's BIOS does this and the resulting menu allows you to select from all currently detected USB drives instead of just the first found to have a boot loader. It may do the same for PATA/SATA drives but I can't test this as the rest of the motherboard only supports a single such drive.