Yes, it can be done. But it's not likely something you would want to do.
A CD carries 80 minutes of stereo sound. But stereo means there are two channels. If you wanted to get creative, you could encode half of your collection as mono in one channel, and half the collection as mono in the other channel. Then, in your CD player, pan all the way left to listen to one channel, and all the way right to listen to the other channel.
This is, of course, quite complicated. I presume you will want to have each file as a separate track, so you'll need to match up similar length tracks, and likely put up with some silence one side as the song on the other channel is completing. You might be able to play around with having more than one track per song/file, if you use the "disk at once" option in your CD burner to eliminate any gap between the tracks.
There won't be any software to help you do this, either. You'll just have to use an audio editor to make the tracks mono, and then put them together in a single file per track, and then burn that file as audio to your disk.
So, while it technically can be done, it is complicated to do and produces an inferior result that has only mono sound and requires a CD-player with panning controls (or disconnecting a speaker).
Your best bet is to see if an MP3 CD (i.e. just burn the MP3s as files, rather than as audio) works on your player, or to just make two CDs.