ALSA is the kernel level sound mixer, it manages your sound card directly. ALSA by itself can only handle one application at a time. Of course, there is 'dmix', which was written to solve this problem. (It's an ALSA module.)
PulseAudio is a software mixer, on top of the userland (like you'd run an app). When it runs, it uses Alsa - without dmix - and manages every kind of mixing, the devices, network devices, everything by itself.
In 2014, you can still run only ALSA. But unless you compile your applications for yourself and enable ALSA support everywhere - or use a source-based distribution like Gentoo - you might get mixing problems. Pre-compiled applications that distros ship are usually only built with support for Pulseaudio, not pure ALSA. Ubuntu for example prefers PulseAudio. It comes with PulseAudio by default, so every application is compiled to only use PulseAudio.
PulseAudio does have its benefits. People say that it is good for working with audio across a network, and it solves some issues with multi-channel audio streams that happened under pure ALSA. It's also supposedly easier to develop apps for PA. On the end-user side of things, it's easy to select new devices, to control volume by app, etc.
However, in the default configuration it adds a not insignificant amount of latency into the mix. This is a big con for certain types of tasks that require low latency like some games and software.
OSS is an alternative to both of these, but it's not licensed under the GPL, which makes it not likely to see a lot of adoption by distros.
Typical PulseAudio-powered sound systems, like Ubuntu:
- Kernel: ALSA -> Userland: PulseAudio -> app1, app2, app3
- In the typical Linux system, PulseAudio mixes audio from all your different apps and feeds them up the chain to ALSA.
- Kernel: ALSA -> dmix -> Userland: app1, app2, app3
- With just pure ALSA, you need dmix to mix multiple apps. Without it, ALSA can only play an audio stream from one app at a time.
- Kernel: OSS -> Userland: app1, app2, app3
- With OSS, the userland apps talk directly to OSS in the kernel, which mixes the streams itself.
So to sum up, in your typical system these days, ALSA talks directly to your sound cards, and Pulseaudio talks to your apps and programs and feeds that into ALSA.