You can copy the keys themselves over using any method, even copying the contents of the keyfile and pasting it into the 'authorized keys' file on the remote server will work.
I wouldn't recommend copying ~/.ssh over, better to just copy only what you need.
Edit...to clarify...
Copying the whole .ssh folder will result in moving the "known hosts" and "authorized_keys" directories onto the remote server, and sometimes the keys are stored elsewhere anyway. Generally, this is not good practice as it makes it much harder to keep track of what has access to what etc. Find the key you need to copy over by checking the IdentityFile line in /etc/ssh/ssh_config, then simply paste the contents into ~/.ssh/authorized_keys file on the remote server.