Unfortunately, you most likely cannot do anything about this. The good news is that you have not been hacked.
Despite my job as a professional IT guy, I knew a man, 25 years my junior, through chess tournaments, who was one of the world's most prolific spammers (before the CAN SPAM law) until AOL sued him out of business. I got the rare opportunity to see how a major spamming operation works, and frankly it was quite amazing. It also gave me great insight into how they operate, although I am sure it has evolved a lot since, although not for him.
He had a T-3 and 40 more 1Mb DSL lines all pumping out millions of SPAM messages daily (250,000/hr 24/7). One of the things spammers do is spoof headers, meaning they say that it is coming from anyone they like. I do a lot of e-mail server work, and part of what I often have to do is test e-mail using Telnet. I put in the e-mail commands line-by-line, and as part of that, I could send an e-mail as anyone in the world, even the President of the United States. There is no verification check.
We can tell that a header is spoofed by looking closely at the header information when the e-mail comes in, and what really counts is the underlying IP address it originated from. Of course, if you had their IP address, you could track them down in no time, so they also spoof that in a way: They relay mail through computers that are infected with viruses designed just to allow this relaying. These spammers find underground programs to scan for these open relays, or often they just pay the owner of a botnet.
So back to how and why you get the bounce messages: I know my acquaintance would just randomly pick roughly 40-50 unlucky e-mail addresses per spamming computer out of his list of millions, and put them in the "From" box of his spamming programs. So any time one of those e-mails gets sent to an undeliverable e-mail address, the bounce message goes right back to the unlucky e-mail address that was used in the forged header.
The only thing you could do is turn off or block all bounce messages, but this is something that is really done at the server level, a level most people do not have access to.
The good news is that they will probably randomly pick different e-mail addresses next time, so the problem will hopefully go away...for you.
For anyone trying to fight spammers, I do not think you can really fight them by trying to track them down using IP addresses, but one thing that struck me watching the operation I mentioned: Follow the money. That is how to really track someone.