.vdi file is "just a file" as far as the host operating system (I'm assuming Windows, here) is concerned. Don't know how much you've learned about operating systems so far, but if you are familiar with the concept of virtual memory and how on-disk files are paged into memory, this is basically saying:
- If you disable host I/O cache, then when the guest does page reads/writes, if it page faults in the guest (the guest doesn't have the page in memory), the page fault will always page fault the host, which hits the storage media with a physical read or write.
- If you enable host I/O cache, then the host will cache pages of the VDI file. Then, when the hypervisor requests these pages from the host operating system, they may not always page fault, if the pages are already in RAM when being read / written.
Writing is a bit more complicated, because it depends on whether your caching strategy is writeback/writethrough, and whether your guest and host filesystems do write combining, delayed allocation, journal/metadata delayed allocation, and so on.
To put it in layman's terms: using the host I/O cache will improve performance, possibly at the detriment of data safety, depending on the design of the host filesystem and its caching strategy. The performance gain is directly related to the amount of "free" RAM on the host (the amount of RAM that is not being used for pages of data resident in an application's virtual address space, or the kernel's address space). More RAM = more performance. This is because, without host I/O caching, the amount of RAM you can use to avoid page faults is limited to the amount of RAM allocated to the guest -- the guest's filesystem / virtual memory manager will do as much avoidance of page faults as possible, but the guest always has less allocated memory than the sum of the guest's memory and the host's memory. When you enable host I/O cache, you get the rest of the host memory too.
Just make sure that, if your USB device is configured for high performance and low data safety (it's an option in the device manager), that you don't "yank" the drive without gracefully telling Windows to remove the drive using the system tray. If it's configured for "quick removal", then that will greatly reduce performance, but you'll be able to safely remove it at any time with almost no loss of data. But always shut down your VM before removing the flash drive.
Another thing: Flash drives are much, much slower than SSDs, as you probably know. The reason is that they are designed cheaply with a single cell, whereas SSDs are basically "lots of flash drives put together" (for higher performance). You get what you pay for. Flash drive performance is always going to be unacceptably slow compared to the performance of a hard drive or SSD, regardless of what connector you use.