Ah... the good ole days.
A: was the first disk device, B: the second, and so on - in CP/M. As somebody else posted, this ran on the 8-bit 8080 and Z-80 machines that pre-dated MS-DOS.
MS-DOS in turn was a 16-bit (8086) knock-off clone of CP/M, or more specifically, CP/M-86, so it used the same drive letter conventions.
Back in those days the only conventions that were pretty much universal were set by CP/M (for example, the disk naming on the Apple-2 and Tandy TRS-80 was something different again... I once used a TRS-80 with 4 floppy drives... oh the power!).
With something like CP/M the first hard drives, when they came out, just appeared as the next available drive letter.
MS-DOS allocated 2 floppy drives pretty much universally and placed the hard drive at C:. Either you had two floppy drives, or the system "virtualised" access to a single drive - an application could access A: or B:, and the system would ask you to swap the physical floppies in the solo drive as necessary. This both increased software compatibility for cheaper systems (floppy drives were not exactly cheap, and software generally used a separate disk for the software, and another for your user data) and made e.g. copying floppies a lot more convenient (still annoying, mind...). This was still back in the day when hard drives were ridiculously expensive (a 6 MiB hard drive would cost around $5k of then-dollars, and a floppy drive around $1k) - most PCs didn't have one.
This convention has been preserved since, though as pointed out here numerous times - in windows - you can change this. As far as I know in MS-DOS you can't.
Slightly off topic:
Back in the 1980's it was pretty common on these early machines for disks to be a mixed bag of different types with legend incompatibilities. The IBM-PC imposed a bunch of standards with the format of its 5.25 inch floppies - before that you had all sorts of different arrangements with hard and soft sectoring, number of sectors / track, number of tracks, etc etc.
Data interchange between machines using floppy disks was a very hit and miss affair, which only got sorted with MS-DOS and the IBM-PC. The most reliable way to transfer files between machines back then was to write some programs (in assembler) to to file transfers using the serial port, and then wire them up. It was never very slow, because files were never very big.
My favourites from then were the high density drive my employer bought - it was an 8 inch floppy attached by a huge long ribbon cable. I think it stored about 200K, which was huge when developing software on a machine with 2 floppy drives, each of them 89K. Compiler on one, source and object code on the other. A compilation took minutes while the drives clanked away.
At the time the IBM PC was introduced you could get it running either CP/M-86 or MS-DOS, and there was no clear indication that one or the other would win. Eventually MS-DOS won the day - some bundling deals might have had a bit to do with that. The first IBM machine my employer bought did make things a lot easier... and the port of stuff to MS-DOS was pretty easy - mainly because of the same conventions for disk drive letters, and also because the system calls to read / write files and populate file control blocks (remember FCBs and the weird format they had in memory?) was the same.
C: wasn't rigid in MS-DOS as the first hard drive. Heath/Zenith MS-DOS had AB reserved for 5" floppies and CD reserved for 8"floppies.
The first default hard drive partition was E, Whether you had floppies installed or not. Their version also allowed 16 partitions on a single hard drive.
I think they may have been the first multi-booting systems as well. Hot-keys during the boot process would allow you to boot from any drive or partition, allowing custom environments and launching CP/M-85, CP/M-86, MS-DOS, Concurrent Dos or HDOS.