The easiest thing to do is change the boot order in the bios so that HDD is first.
The computer won't look for other bootable devices once it's started booting from the HDD.
EDIT: You've now mentioned you cannot change the BIOS settings (or don't want to).
It's unusual that it sounds like it even attempts a boot process. You haven't been very clear on what you're seeing on screen. It shouldn't start booting a flash disk with just straight partitions on it. I suspect you have a partition maked bootable and an MBR or something on there that thinks it should boot from it. Have you had an OS loaded onto it before.
Otherwise, the BIOS initiates boot. The first thing that the computer will do is run the BIOS firmware. The BIOS may look at the flash disk. It should pass over it if it's not got an MBR record So unless you instruct the BIOS NOT to boot from a flash disk, nothing you do to it will make any difference. The BIOS may look at the disk to see if it has an MBR or similar and if it doesn't it should skip it moving onto the next device in the boot priority.
Someone has suggested pressing a key combination to select which device to boot first. However, IMHO that's just as much trouble as unplugging the drive and plugging it back in.
Can you check the drive doesn't have a partition marked bootable, and there is no MBR on there despite there being no OS is there? It should just be a straight partition.
EDIT2:
Since the computer is hanging at "verifying dmi pool data". The following link may be of benefit.
Google: "verifying DMI Pool data freeze"
It's an extremely common problem.