Ubuntu isn't as tacked down to hardware as much as many other linux distros. One example of this is take a look at the amount of kernel modules. The number Ubuntu uses is completely unnecessary, but they need to all be there if you want to be able to plug in a new piece of hardware without having to recompile your kernel. You could actually install to a harddrive, take that harddrive out and move it to a different computer and boot that, and you would have much the same success that your seeing with your thumb drive.
There are, of course, many things that aren't going to be that easy. The two primary examples that come to mind are video drivers and wireless drivers. If you install Nvidia drivers on one computer with an nvidia video card, and switch to a non-nvidia computer, you may run into issues. I looked through a lot of the Ubuntu documentation for references to USB installs, and none mention any differences to a harddrive install. I really don't think there will be any unforeseen issues from doing the install on one computer and moving to another apart from the normal things you'd have to do to get up and running (like installing the correct video drivers).
You know, there are other options too. If you're just trying to install Ubuntu on a computer without a CD drive you can install FROM a USB stick. Or, if you're trying to run Ubuntu from a computer without a harddrive you could setup a network boot where all the data is stored on the network and the computer can remain harddriveless. Running an OS off a thumbdrive isn't necessarily the best option because thumbdrives do have a limited life. They can only handle so many read/writes before they break down... far fewer than a regular harddrive... This is why or Solid State Harddrives are so much more expensive that flash thumbdrives, even though they are very similar. Harddrives have to support far more read/writes among other things. Talk to anyone who uses a thumbdrive many times a day and you'll see what I mean. Many wear out in a matter of a year or two. An OS does a lot more read/writing than just placing regular files on and off the drive... Things like SWAP are in constant flux.