If you were to close that program, you would only be closing the control panel. The drivers for your display adapter would still be loaded (in fact, they are loaded at boot, instead of being loaded at Windows start, like the nVidia control panel is). So, closing it would not do anything but free up a little bit of memory on your computer.
If you were to unload the drivers for the display itself, you would simply see Windows revert back to the default, stripped down VGA driver, and you'd just lose all your 3D acceleration and higher resolutions. But to do that, you would have to go under the device manager, where it will let you click "uninstall driver for this device".