That's quite a big question =) The below is a high-level overview of HTTPS; shout up if anything isn't clear and I'll fill in more details.
When you use an HTTPS connection, public key crypto is only used during session initialisation. Part of that initialisation is the secure swapping of a private, shared key that is then used for symmetric encryption. The reason for this is that asymmetric (public key) encryption is significantly slower than symmetric key encryption, due to the maths involved.
At a very high level, the protocol goes something like this:
Client sends initial "HELLO" message, that contains the ssl version it would like to use, a pseudo-random string of bits (let's call that B1) and a list of the various ciphers it supports.
The server responds with with its own "HELLO" message, another pseudo-random string of bits (let's call that B2), the cipher it has selected (normally the strongest cipher from the list the client sent that the server supports), and its certificate which contains its public key (let's call that k).
The client then uses the certificate to authenticate the server (see below) and creates a "premaster secret" - another string of pseduo-random bits that we'll call S.
If the client is happy with the results of server authentication, it encrypts the premaster secret S with the server's public key k, and sends it to the server (S itself is encrypted, but the connection is still not).
At this point, the server and client share three strings of bits - B1, B2 and S. B1 and B2 were sent in the clear - S was encrypted, and so, assuming the server's private key really is private, is known only to the client and server.
Both client and server then use these three bit strings to construct the session key - a string that is known only to them, and can be used as a key in the previously selected symmetric cipher.
Client and server then exchange messages that indicate they are changing protocol, and all subsequent messages are encrypted (symmetrically, with the key computed above).
When the client receives the server HELLO message, it needs to be sure that it is talking to the server it thinks it is. The server cert and key are used to establish this trust.
All a digital certificate is is a third party asserting "this entity holds the private key that corresponds to this public key, which is embedded in this certificate". Organisations like Verisign will make this assertion - for a price. The reason people pay Verisign and others for certs is that Verisign have gone to the trouble of getting their intermediary certs embedded in most of the common browsers.
So, when the client receives the server HELLO, it will perform the following checks:
- Does today's date fall within the certificate's validity period?
- Does the subject name of the certificate match that of the server I am trying to connect to?
- Does the Issuer name on the certificate match any of the CAs whose public keys I know about?
- If so does the public key I have for this CA validate this certificate?
If all those checks pass, the client assumes it can believe that the server is who it was expecting. It then takes the public key (still k) out of the cert, and uses it to securely transmit S to the server. At this point, the idea is that you have some third party's (Verisign's) assertion that the public key belongs to the server; so only the server will be able to decrypt the result of E(S,k), and hence only the server can produce the matching key that you'll use for the symmetric cipher.
After the handshake, your confidence that the packets cannot be read by a third party should be equal to your confidence in the symmetric algorithm that's being used.
(There are other nice twists - eg the reason three bit strings are used is to prevent replay attacks. If only S was used, an attacker could record the entire session and replay it at leisure - for example, repeating a money transfer instruction over and over. By having both client and server generate additional pseduo-random strings, you greatly reduce the likelihood of two independent SSL handshakes producing the same S.)