First, you are talking about Trojan's, not viruses, which is an important distinction for your particular question.
A drive-by attack downloads a malicious file to your computer, and then executes it. upon executing, the trojan performs its bad actions.
Since a windows trojan will not be able to execute on the Linux kernel, it will just sit there taking up space exactly as you suggest. in this state it is not dangerous.
If a windows OS were to access and execute the trojan however, (perhaps over fileserver share connection as Rich suggests) the windows machine may become infected.
Note however that the trojan will not 'activate' itself, unless run. there is a minute possiblilty that your windows system may autorun the trojan when you plug the flash drive in, so be sure your autorun/autoplay features are set cautiously.
Other than that, using some common sense and healthy skepticism your risk is minimal. Don't click exe's that you don't know, especially if you haven't scanned them with an updated AV system (though AV is far from a guarentee that the file is clean ).