It depends on you where you want to install the new Ubuntu system. Bsically, there are two critical points.
First, you will be asked to chose on which medium to install Ubuntu. You will be offered the choice between the HDD and the USB stick. Make sure you pick the right one. One way to make sure is to look at available media before installing, i.e. in the phase Try Ubuntu without installing. Open a console, then issue one of two following commands:
sudo fdisk -l
sudo gdisk -l
(Normally you will need the second if and only if the HDD has a GPT partition table). In any case, these commands will present enough info (size and partition table) about the disks to allow you to identify the HDD and the USB sticks. Please keep in mind that data pertaining to the installation USB stick will also be presented, so that actually you will have three disks (one HDD and 2 sticks) for which to determine whether they are dev/sda, /dev/sdb, /devsdc.
The second point is where you want to mount GRUB, which depends on the use you intend to make of the USB stick. Do you plan to use you it as a standalone on other pcs as well? Then you should make sure that GRUB makes no reference to any OS on the HDD, and you should have it mounted on the USB stick. The second part is easy: during installation you will be asked where you want GRUB, just make sure you answer with the /dev/sdX for the USB stick. As to the first part, after installation you can remove any reference to OSes on other disks by following any of the methods suggested by Pro BackUp in this unix.stackechange.com answer. Now, in order to boot from the USB stick, you will have to go to the BIOS and choose the USB stick as a higher priority than the HDD.
If instead you think you will always be using the USB with this pc, you may as well reply that you wish GRUB to reside on the HDD. This way you will not have to go into the BIOS to select Ubuntu over Windows, and you will be presented with the choice of OS to boot at every boot.