is there any chance that I could
download true minimal drivers
Not very often, unfortunately. However, most of the big driver packages can be extracted so that you can get at just the driver portions. It really varies from vendor to vendor. Some of them have command line switches for extracting. Some of them are just self-extracting zip or 7-zip files.
Some of them are annoying setup.exe's. But even with those, you don't have to finish the setup. You can start the installer and go rooting around in your temp directory for the extracted installer files. Many times the drivers will be in there and you can copy them out before canceling the setup.
The worst offender in my experience has been HP printer drivers. I had to download a driver package from them one time for an ink jet printer that was, no joke, 300MB....for a freaking printer!
How and where can I download true minimal drivers...
This is an update for your edited question. I hate to break it to you, but you can't. There's no magical site that I know of that archives hardware drivers from every imaginable hardware vendor on the planet. Your only reliable source for hardware drivers is from the vendor of the hardware or the vendor of the parent chipset (in the case of things like video cards). Trust me, I deal with enterprise windows deployment for a living on many different models of PC. Dealing with these driver packages is just part of the job.
About the closest you can get to what you're looking for is DriverPacks.net. But it's not a site that you can search for your hardware and download the individual driver. They deal only in sets of drivers and they're aimed primarily at people creating custom Windows installation media. You'd likely have a harder time trying to scour through the packages looking for your driver than you would just dealing with the individual package from your hardware vendor.