This is probably expected Windows behavior, but there are possible work-arounds.
Explanation: The input language (keyboard layout) is not global; it is set on a per-application basis. When an application loses input focus, Windows stores the current input language. When that application regains input focus, Windows tries to switch back to the input language it previously stored for that application. I am not sure about middle click, but the Windows key and Alt-Tab definitely induce the input focus to be switched to a different application, and thus potentially switch the input language.
Here are possible work-arounds:
If you use only two layouts and one of them includes a key for switching to/from QWERTY (like Korean and Hebrew), uninstall all other layouts. Just use the single layout's ability to switch to/from QWERTY. Note the layout's QWERTY input state is still stored and automatically switched between different applications. I've used this method to switch between QWERTY and Korean input in a sane, predictable way.
3rd party software solution (free):
Portable Keyboard Layout (PKL) is used for supporting alternate input languages on (other) computers without admin access. It maintains the same input language state across all applications. A keyboard shortcut for switching between layouts can be configured. Works from a USB drive. Free and open-source.
Some light setup is required:
- Copy desired layouts to layouts sub-folder
- Add layouts and keyboard shortcuts to the pkl.ini file. I found the Hebrew layout for you here. (Be sure to remove the Colemak layout if you are not interested in it.)
Here is a good start on how the pkl.ini file should be modified:
; The layout ini file is "layouts/[key value]/layout.ini"
; You can use multiple layouts, if you write:
; layout = layout1:Displayed name,layout2
layout = hebrew:Hebrew
; With this hotkey you can switch between layouts
changeLayoutHotkey = LAlt & LShift
The above settings add the Hebrew layout and the Alt + Left Shift keyboard short-cut you are used to (I would uninstall all other layouts and keyboard shortcuts on Windows so they don't interfere, or use different keyboard shortcuts)
As a side note, I modified the PKL script to support the odd combination of Korean Hangul input and Colemak layout. With the default PKL install, Korean input is horribly broken because it expects a QWERTY layout...