I'm trying to set up a RADIUS server for WPA2 enterprise authentication. I'm new to all this. I understand that the server has a "server certificate", which I assume plays a role similar to that played by HTTPS and SSH server certificates. Does there exist a concept of "client certificates"? Is that a meaningful notion here? If so, how are they used and what purpose do they serve?
The Short Version
Yes, there's a concept of "client certificates" or "user certificates". They actually exist in the TLS (TLS being the modern successor to SSL, the security scheme for HTTPS) world too, but they're not in common use there.
EAP-TLS is the authentication method within the WPA2 Enterprise/802.1X/EAP/RADIUS world that uses certificates for authentication in both directions. It's the authentication parts of TLS packaged as an EAP authentication method.
Because of how difficult it is to provision and manage certificates for every user or every client machine, most organizations don't use user certs for authentication. But note that "Smart Card" authentication actually uses user certificates behind the scenes. So if your org already issues Smart Cards to everyone, EAP-TLS is for you. This is why the Windows UI for selecting EAP-TLS (in the Windows Wi-Fi client UI) calls it "Smart Card or other certificate".
Some sites use TLS client certificates as "machine"/"device"/"system" certificates, so that a machine can get onto the network without user interaction. This is useful if you want the machine to stay on the wireless network for remote management and backups even when no user is logged in.
The Long Version
If you know about HTTPS, then you know it implies SSL or TLS, and you probably know that TLS is the modern successor to SSL. You already probably know about how TLS uses X.509 server certs to allow a client browser to authenticate a server, but did you know that TLS can use certificates in both directions? Yep, you could issue X.509 certs to your users and have them install them on their client machines (or in their browsers), and then your web server could authenticate your users via certificate instead of password.
If you've ever used "Smart Card" authentication, you've actually used X.509 user certs behind the scenes without realizing it. Smart Cards allow user certs—especially the private keys that mate with the public keys in the certificates—to be stored securely within the card.
So now let's talk about WPA2-Enterprise. WPA2-Enterprise uses a technology called "Extensible Authentication Protocol (EAP) over Local Area Networks (LANs)" or "EAPoL". EAPoL was standardized in IEEE 802.1X, and applied to IEEE 802.11 along with other security fixes and updates in IEEE 802.11i. Then the Wi-Fi Alliance went and created a certification and logo licensing program based on 802.11i, and called it "WPA2", and when you use the optional 802.1X parts of WPA2, it's called WPA2-Enterprise.
EAP pre-existed EAPoL. EAP was a technology that came from the PPP dial-up days, when the built-in authentication methods in PPP were kind of weak and hard to modify, so the industry came up with a kind of pluggable authentication scheme for PPP and called it EAP. RADIUS was already in use as a way to keep PPP credentials on a central server, so RADIUS servers formed the "authenticator" (server) side of EAP. By the way, since many VPN technologies use PPP inside an encrypted tunnel, EAP has been common in the VPN world as well.
So when you hook up WPA2 Enterprise to RADIUS, you're really hooking up 802.1X (EAPOL) to RADIUS, and doing an EAP-transported authentication between the Wi-Fi client and the RADIUS server.
Since EAP is extensible (pluggable), there are many authentication methods (EAP methods) you can plug into EAP. For example, if you like the authentication parts of TLS, you can use those in EAP, and it's called EAP-TLS. Or you can use a EAP method called "Protected EAP" (PEAP) which is really "EAP-within-EAP". As the inner EAP method in PEAP, many people choose to use EAP-MSCHAPv2 or EAP-GTC, but some people use EAP-TLS inside.
Lots of companies, most notably Microsoft and Cisco, have tried to make it easier to get user or machine certificates onto users' devices, so there are ways to set up your Windows Server environment or your Cisco network environment such that the network tries to push certificates onto your users' machines when they first connect to the network. One such effort is called SCEP, the "Simple Certificate Enrollment Protocol".
Some sites like to have ways for a laptop to be securely authenticated to the Wi-Fi network even when no user is logged in (for the purpose of system management, unattended overnight backups, etc.). Since no user is logged in, the users' certs (or at least their private keys) are not accessible to the system. So Windows machines, Macs, most smartphones, and other devices have ways to install a "machine/device/system" certificate onto the machine, so that the machine uses its own certificate (not any of its users' certificates) to authenticate to the network when no user is logged in. Some sites even use machine certificates to authenticate the machine to the network even when a user is logged in, because they don't want to bother their users with having to log into the Wi-Fi network themselves.