Following this question I'd like to try and completely remove the Qwerty layout. However, the default tool that comes with Windows won't allow such change. I wonder if there is a lower-level way to achieve this.
Open Control Panel then "Region and Language".
Next, go to the "Keyboards and Languages" Tab.
Simply click on any keyboard and as long as you have a secondary keyboard, you should see the option to remove.
The key issue is that the one you want to remove cannot be the current default. Take note of the top group labeled: 'Default Input Language' and ensure that the one you want to remove is not the default. Once you correct that, the remove is no longer grayed out.