Nearly 2 decades ago would have been dominated by the range of Windows 98 through XP, including NT4 and 2000 on the workstation/server side.
All hard drives would also be PATA or SCSI cabled magnetic storage, as SSDs cost more than the computer, and SATA did not exist.
As WooShell's answer says, the lower logical sectors on the drive (outside of platter) tend to be the fastest. My 1TB WDC Velociraptor drives start out at 215MB/s, but drops down to 125MB/s at the outer sectors, a 40% drop. And this is a 2.5" drive platter drive, so most 3.5" drives generally see an ever larger drop in performance, greater than 50%. This is the primary reason for keeping the main partition small, but it only applies where the partition is small relative to the size of the drive.
The other main reason to keep the partition small was if you were using FAT32 as the file system, which did not support partitions larger than 32GB. If you were using NTFS, partitions up to 2TB were supported prior to Windows 2000, then up to 256TB.
If your partition was too small relative to the amount of data that would be written, it is easier to get fragmented, and more difficult to defragment. Of you can just straight up run out of space like what happened to you. If you had too many files relative to the partition and cluster sizes, managing the file table could be problematic, and it could affect performance. If you are using dynamic volumes for redundancy, keeping the redundant volumes as small as necessary will save space on the other disks.
Today things are different, client storage is dominated by flash SSDs or flash accelerated magnetic drives. Storage is generally plentiful, and it is easy to add more to a workstation, whereas in the PATA days, you might have only had a single unused drive connection for additional storage devices.
So is this still a good idea, or does it have any benefit? That depends on the data you keep and how you manage it. My workstation C: is only 80GB, but the computer itself has well over 12TB of storage, spread across multiple drives. Each partition only contains a certain type of data, and the cluster size is matched to both the data type and the partition size, which keeps fragmentation near 0, and keeps the MFT from being unreasonably large.
The downsize is that there is unused space, but the performance increase more than compensates, and if I want more storage I add more drives. C: contains the operating system and frequently used applications. P: contains less commonly used applications, and is a 128GB SSD with a lower write durability rating than C:. T: is on a smaller SLC SSD, and contains user and operating system temporary files, including the browser cache. Video and audio files go on magnetic storage, as does virtual machine images, backups, and archived data, these generally have 16KB or larger cluster sizes, and read/writes are dominated by sequential access. I run defrag only once a year on partitions with high write volume, and it takes about 10 minutes to do the whole system.
My laptop only has a single 128GB SSD and a different use case, so I cannot do the same thing, but I still separate into 3 partitions, C: (80GB os and programs), T: (8GB temp), and F: (24 GB user files), which does a good job of controlling fragmentation without wasting space, and the laptop will be replaced long before I run out space. It also makes it much easier to backup, as F: contains the only important data that changes regularly.