Yes, there will be a difference.
As far as I am aware the faster USB flash drives tend to be around 20MB/s write speed (but speed varies a lot depending on manufacturer and controller) while hard drives tend to pretty much saturate the USB port and transfer near the limit at around 40-50MB/s
Hard drives are much faster at bulk data transfers, while Flash devices tend to be much faster at finding small bits of data quickly.
-=EDIT=-
To answer your comment
You can get some portable 500GB drives that will work from USB bus power alone, they tend to be based on laptop hard drives and can be a bit slower than the full size hard drives, but generally they're fine. An example is this one.
A full format can take quite a while and it only takes longer the larger the drive is, with an approximate 500GB drive I would expect a full format to be of the order of 3 hours at USB speeds
(500*1024)Mbytes / 40MBytes/s = 12800 seconds
= 213 minutes
= 3.5 hours (approx)
It may end up being a bit faster (something like 2.5 hours) due to getting slightly better speeds (USB can theoretically get up to 60MB/sec, but there are protocol overheads and other devices that share that speed), manufacturer megabytes (as manufacturers do not measure a megabyte the same way everyone else does) and other factors. Heck, it may even take longer if it's a particularly slow drive or the USB bus is doing other things....
Generally though, USB drives come preformatted, have protection (S.M.A.R.T.) so that if they fail to write a sector they will remap the sector out of the usable portion of the drive, and a quick format will suffice 99% of the time.