This isn't quite impossible to answer, but it's practically very difficult to do as it varies based on the data you're going to write to a disc. Basically, you'd need to do what CD/DVD burning software does to determine whether you've exceeded the capacity of disc before burning. [And I'm sure some, if not most, such programs fail to do so perfectly.]
Approximately, what you've already done, i.e. restrict yourself to a smaller capacity, is probably going to be the only feasible solution in general.
The program IsoBuster is actually pretty neat. You can pretty easily see exactly how the files and directories are stored in a disc image and you could probably mostly reverse-engineer how a particular filesystem allocates space for different types of files using it. I'm going to try that myself.
Like you, I discovered that you can't simply compare the total size of the files in bytes and the total "gross capacity" in bytes of a disc.
Firstly, filesystems, both on a hard drive and on optical discs like CDs and DVDs, require some space for data about the files, e.g. the directories in which they're stored, the name of the file, when it was last modified, etc.
Every filesystem is (almost certainly) going to require different amounts of space for different sets of files and directories; they're also going to have different restrictions in terms of things like the maximum length of file names, whether file names can be made up of Unicode characters, etc.
Different filesystems also support or require different sector/block sizes, which is roughly the smallest amount of space (almost always expressed in bytes) that can be allocated to a file, i.e. a 512 byte sector size implies that every file is allocated at minimum 512 bytes (for the file data alone). However, for the UDF filesystem, which can be used for CDs and DVDs (and Blu-ray discs):
Small files (and directories) can be stored in the File Entry block itself, similar to the embedded files in NTFS.
i.e. small files (and directories) can be stored in the 'filesystem metadata' itself. [The source is resource  below.]
- Understanding DVD - Disc Size, Configuration and Capacity
- Universal Disk Format - Wikipedia, the free encyclopedia
- Universal Disk Format® Specification, Revision 2.50
- Wenguang's Introduction to Universal Disk Format (UDF)
- CDBurnerXP: Create Data Disc
- Disk sector - Wikipedia, the free encyclopedia
- Optical disc authoring - Wikipedia, the free encyclopedia
Even if you don't add any files, the used disc space will be somewhere above 1MB. This is because every data image has an inherent system overhead for the allocation tables, volume data, system space, and path tables (to name a few). Also, some space needs to be reserved for lead-in and lead-out processes, approx. 13 MB as of version 4.5 of CDBurnerXP.
Based on , I don't think the "lead-in" and "lead-out" necessarily applies to data discs.