While installing ubuntu, there is a high possibility GRUB was setup on the computer's internal hard drive. Grub requires some files to load properly and in this case, these files would be on the external hard drive.
I would recommend re-installing the basic windows MBR on the computer's hard drive and then re-setup GRUB on the external hard-drive. You can then configure your bios to first boot on the external hard drive. If connected, it will properly load GRUB, otherwise will fall back to the internal one.
You can reinstall the MBR if windows still boot by opening a shell and typing:
bootsect /nt60 C:/
The instructions are usually for computers no longer booting at all.