I recently bought two Hitachi HLSMPUA3201ABB external Hard drives. They come with a USB key attached to the enclosure with a magnet. It's not a very strong magnet but I can't help wondering whether the drive would last longer without it. Should I consider returning it?
It won't damage your HDD, don't even give it a second thought.
See here for reasons why.
There are magnets inside the hard drive also and are not shielded in any way.