From my understanding, we use VMs for isolation of environments
… so that one environment's configurations does not break another environment.
That's too narrow. Isolation is about security in a broad sense: confidentiality, integrity, and availability.
- Confidentiality: isolation provides fine-grained permission management. VMs and Docker container can only access data and other services you explicitly let them access.
- Integrity: services themselves can be decoupled from logical and physical storage mechanisms.
- Availability: Individual services can be reconfigured, updated and restarted independently from each other. Crash of one service doesn't affect other ones. Resource management is also more flexible: you can split one physical machine's resources across many services and manage allocated resources on the fly. Doing this on actual hardware isn't that easy, you can't hot-swap a CPU to a more powerful one (or less powerful one, if you want to allocate more resources to another service).
With such a big performance cost, wouldn't it be better to actually come up with a better operating system that actually has built-in isolation of environments?
Modern CPUs support virtualization, for example look up VT-x available in Intel CPUs. It lets CPU run virtual machines without any performance impact.
Docker saw that and that's why they came up with containers
Docker doesn't implement containers, it only uses containerization features provided by the host OS. It's just a convenient way of configuring them to work together and to share pre-built service images. To name just a few OS-provided features, on Linux Docker uses overlayfs, cgroups and iptables. You can use all of these without installing docker. So Linux has built-in isolation of environments, Docker just makes it easy to use.
It may be worth to mention Android here, which, while based on Linux kernel, uses the the ART VM runtime and provides stricter isolation of apps than "regular" GNU/Linux does by default.
Why aren't OSs just done in a way that virtualization wouldn't be needed?
Initially they weren't built with as much isolation in mind and we all got used to this model. While less secure, it's also more comfortable and less cumbersome. It's easier to just let all apps do everything and not manage fine-grained permissions manually. Lack of strict isolation is the reason both for viruses' abilities to do much damage and for MS Word's legit ability to save a document to any folder on your hard disk.