Does a software like VMware run in user space or kernel space?

Question

Vmware is a user installed software. So it should run in user space given by the underlying OS. Yet its runs OS-es which are fully operational and need to run in kernel mode. So where does it run?

Answer 1

VMWare provides a whole bunch of projects and Vinko Vrsalovic already pointed out that VMWare ESX is indeed an own operating system.

Generally, it is possible to run virtual machine monitors (VMM) such as VMWare Workstation, QEmu, or Virtualbox completely in user space. They just take a disk or CDROM image of the OS to run and start emulating a PC by interpreting all instructions and emulating device accesses. For performance reasons, virtualization software however tries to avoid emulation as often as possible, because it is much faster to execute an instruction stream on the real hardware instead of emulating it.

Several approaches are used to speed up virtualization process:

• Binary rewriting: Your virtualized OS executes an OS kernel and user space applications. For user-space apps, the same restrictions in terms of instruction execution apply as do to your virtual machine monitor itself, so it is no problem to execute these applications on the native CPU. For the OS part, you however need to emulate the behavior of the underlying CPU as your VMM is not able to do so itself. What you now need is to detect all switches between user code and kernel code, so you can efficiently switch between native execution and emulation. VMWare as well as Qemu apply binary rewriting to the virtualized OS as it is executed and thereby trap all those switches.

• Hardware-assistance: With the rising popularity of VMMs (which have been around since the 1970s), Intel and AMD started providing hardware extensions to their instruction sets which ease implementation of a VMM in user space.

• Kernel extensions: VMWare as well as Qemu also enable you to install a device driver in your host operating system that enables them to speed up execution of kernel code and device accesses by directly executing in kernel mode.

• Pass-through device drivers: Device I/O is another performance issue. Normally, a VMM will implement a set of virtual devices that behave similar to their real-world counterparts. This emulation step slows down I/O and is a major performance bottleneck when virtualizing an OS. To avoid that, VMMs also allow you to install a bunch of device drivers in the virtualized guest OS that are aware of the guest hardware and allow to directly pass through a device from the host OS to the guest without the emulation step. (This however only works if you run exactly one OS instance on your host, because no one else will be able to access the device if you pass it to the guest).

Answer 2

Discounting the fact that software like VMWare ESX is an OS by itself (a modified Linux version), the on top of another OS version usually run in both kernel space and user space, it installs drivers (which run in kernel space) for all the privileged tasks (networking, monitoring and so forth.)

Answer 3

It just depends on what type of VMWare program you run. Some run in the kernel, some run in user space.