Yes, there will be a difference.
As far as I am aware the faster USB flash drives tend to be around 20MB/s write speed (but speed varies a lot depending on manufacturer and controller) while hard drives tend to pretty much saturate the USB port and transfer near the limit at around 40-50MB/s
Hard drives are much faster at bulk data transfers, while Flash devices tend to be much faster at finding small bits of data quickly.
To answer your comment
You can get some portable 500GB drives that will work from USB bus power alone, they tend to be based on laptop hard drives and can be a bit slower than the full size hard drives, but generally they're fine. An example is this one.
A full format can take quite a while and it only takes longer the larger the drive is, with an approximate 500GB drive I would expect a full format to be of the order of 3 hours at USB speeds
(500*1024)Mbytes / 40MBytes/s = 12800 seconds
= 213 minutes
= 3.5 hours (approx)
It may end up being a bit faster (something like 2.5 hours) due to getting slightly better speeds (USB can theoretically get up to 60MB/sec, but there are protocol overheads and other devices that share that speed), manufacturer megabytes (as manufacturers do not measure a megabyte the same way everyone else does) and other factors. Heck, it may even take longer if it's a particularly slow drive or the USB bus is doing other things....
Generally though, USB drives come preformatted, have protection (S.M.A.R.T.) so that if they fail to write a sector they will remap the sector out of the usable portion of the drive, and a quick format will suffice 99% of the time.