On most, if not actually all, current widely used architectures, memory addressing is done in bytes.
A 32-bit address width allows you to specify 232 (about 4.3 billion) addresses.
This means that a CPU that uses 32-bit addressing and addresses RAM in terms of bytes can natively address 232 bytes, which equals 4 GiB (gibibytes) or 4,294,967,296 bytes.
On many current architectures, there are other issues to consider, such as memory request address alignment, which limits what addresses can be used for actual access. But that does not change how much RAM the CPU can address, it only changes how it can be addressed, and unless you are working very close to the CPU this is normally abstracted away from the programmer, and certainly from the user.
It's also worth keeping in mind that the "bitness" of the CPU doesn't necessarily refer to the address width of the CPU, but rather normally refers
to the CPU's native word size. Take the Intel 8088, as used in the original IBM PC, and its sibling the 8086; it had a 16 bit word size, but a 20 bit address width, and was typically referred to as a 16-bit CPU despite the fact that the 8088 only had an 8-bit data bus width (the 8086 had a 16-bit data bus). The Intel 80386SX had a 32-bit native word size but a 24-bit usable address width; the 80386DX had a full 32-bit address width. The Motorola 68000, roughly contemporary with the 8086/8088, had a 24-bit address width, but was a hybrid 16/32 bit design. And so on. The native word size may be the same as the address width, but it does not have to be.
There are ways (such as Physical Address Extension, PAE) to get around the 4 GiB limitation normally imposed by a 32-bit address width, just as CPUs with 16-bit addresses used various techniques (including segmentation and bank switching) to be able to address more than 64 KiB (216 bytes) of RAM. Also, if the addressing was done in a different unit (for example, 16-bit words rather than 8-bit bytes) then the actual amount of memory that could be addressed would change accordingly; as an example, by addressing 16-bit words rather than 8-bit bytes, a 32-bit address width would allow addressing 8 GiB because each address actually addresses two bytes.