Have you heard of asymmetric encryption (also called public-key cryptography)? That's the technology that SSH keys are based on.
The basic concept of asymmetric encryption is that you have two cryptography keys, which are typically very large numbers (or equivalently, long strings of binary data). Any message encrypted with one of them can be decrypted only with the other one - not even with the original encyption key itself! This is useful because you can keep one of the keys, the private key, to yourself, and publish the other one, the public key, somewhere online or wherever you like. People can encrypt messages using the public key and be confident that only you, the holder of the corresponding private key, can decrypt them and read their contents. (This is how GPG and PGP and similar programs work, if you've ever heard of those)
It turns out that the process also works in reverse, with the keys switched. If you encrypt something using your private key, it can only be decrypted using the corresponding public key. This may seem useless because anyone in the world can get your public key and decrypt the message, but it does have one very useful side effect: it confirms that the message came from you. Because nobody else in the world has your private key, nobody else could have made a message that can be decrypted with your public key.
If you think about it, being able to verify that a message came from you and not from some impostor is exactly what authentication is all about. Every time you log into a website using a username and password, you're providing some secret information that supposedly only you know, in order to prove that the login request is coming from you. Well, you can do the same thing with your private key: encrypt your login request in order to prove that it comes from you. And that's how SSH key pair authentication works. Instead of you providing a username and password, the SSH server sends your SSH client a message to be encrypted with your private key. The client encrypts it, sends back the encrypted message, and the server decrypts it using your public key to check it. If it passes the check, congratulations, you're logged in!
I haven't signed up for GitHub myself (yet) so I'm not familiar with the exact procedure, but I suspect that when they ask for a username@host, that's just as a comment to identify the particular key. If you log into GitHub from different computers, you'll have a different SSH private key for each of those computers, and GitHub needs to keep track of all the corresponding public keys. The username@host is a pretty standard way to label SSH public keys that come from different people and/or different hosts.
By the way, it's worth mentioning that SSH key pair authentication is generally considered more secure than username/password authentication. The reason is that, when you use a username and password to log into a website, you have to share that information with at least one other computer: the web server that runs the site. Most of the time, that's fine, but it's always possible that whoever is in charge of the website, or even just someone who has access to the server, could be recording your password and using it behind your back to impersonate you. Using a private/public key pair eliminates that possibility, because you never share the private key with anyone at all, not even the server you're logging into. So, although the server (e.g. GitHub) is perfectly able to check whether it's really you logging in, it still doesn't have enough information to pretend to log in as you.
As for how you create an SSH key pair: I'm a Linux user myself, so I'm not sure how it'd be done with the software you're using on Windows. Whatever SSH client program you use (PuTTY?) should give you some way to create a key pair.
