How can I use Caps as a "function" key to activate arbitrary combinations on other keys?

Question

For context I am trying to replicate my hardware-programmable 60% keyboard experience in software for my laptop.

I am using colemak (setxkbap us -variant colemak) on Arch Linux with i3wm and no desktop environment.

I want Caps to act as LCtrl for most keys on the keyboard (eg Caps+c is LCtrl+c), but activate other combinations for specific other keys, specifically Caps + h/n/e/i for Left/Down/Up/Right as system-wide vim keybindings.

Testing method

• Open a terminal
• Press Caps+e, expect previous command to appear ('up' action)
• Press Caps+c, expect new terminal prompt ('Ctrl+c' action)

What I have tried

Attempt 1

Modify the xkb layout to:

• Assign Caps key to some other modifier (e.g. Mode_switch or ISO_Level3_Shift)
• Assign the keys I want on that modifier layer

Using the following .Xmodmap file, I can achieve half of what I want.

keycode 66 = Mode_switch
keycode 43 = h H Left
keycode 44 = n N Down
keycode 45 = e E Up
keycode 46 = i I Right

This works well for mapping single keys, but there does not appear to be any way to set anything like keycode 54 = c C Ctrl+c

This could also be achieved by modifying the olemak section in /usr/share/X11/xkb/symbols/us, but would be subject to the same limitations.

Attempt 1

Modify /usr/share/X11/xkb/symbols/us to set Caps as Mode_switch and then redefine the AltGr bindings for the relevant keys.

This works for directional keys, but I can't see a way to set the AltGr layer to a combination of keys (Ctrl+c)

Attempt 2 - AutoKey

Set Caps as Hyper with -option caps:hyper on my startup setxkbmap command. Use AutoKey to bind Hyper+KEY to scripts/phrases.

Attempt 2.1 - script send_keys
Bind Hyper+c to this phrase: keyboard.send_keys("<ctrl>+c")
Result: Nothing

Attempt 2.2 - script press_key, release_key
Bind Hyper+c to this script:

keyboard.press_key("<ctrl>")
keyboard.send_key("c")
keyboard.release_key("<ctrl>")

Result: About 1 in 20 times, Ctrl+c actually appears to get sent (the terminal displays a new prompt on a new line). If I keep spamming Hyper+c it will eventually work. Most of the time does nothing.

Attempt 2.3 - phrases
Bind Hyper+c to phrase: <ctrl>+c Result: AutoKey crashes

Attempt 3 - Python

Attempt 3.1 - pynput
Followed this guide about creating hotkeys in Python with pynput. Set Caps as Hyper the same as above. Create binding for Hyper+c, to invoke:

keyboard.press("ctrl")
keyboard.press("c")
keyboard.release("c")
keyboard.release("ctrl")

(I tried various other values etc at the time, that I can't since remember). Result: Nothing

Attempt 3.2 - pynput with pyautogui Same as above, but use pyautogui to send the Ctrl+c combination instead. Result: Nothing

Attempt 4 - xdotool in i3 config

Use i3 bindsym to invoke exec xdotool key ctrl+c Result: Nothing

Attempt 5 - xdotool with xbindkeys

Set Caps to Hyper_L and try the following.

"xdotool key --delay 0 ctrl+c"
    Hyper_L + c + Release

(Also tried with caps as Mode_switch and Mode_switch + c + Release.)
(Also tried with c:66 + c + Release.)

Result: Nothing

Interesting enough I can bind individual keys to 'Ctrl+c' this way, but not the combination of the two.

Attempt 6 - sxhkd

Use sxhkd to replicate the above (don't have the exact syntax to hand right now) Result: Nothing

I don't want to bind Caps directly to Ctrl

One idea would be to bind Caps to Ctrl, then bind Ctrl+h/n/e/i to arrows. I don't want to do this because I want to reserve the ability to actually combine the actual Ctrl key with h/n/e/i for instances when I actually need those Ctrl combinations.

Conclusion

Sorry for not providing exact code for each of these. I've spent and entire day trying to get this to work and haven't meticulously documented every step. Safe to say that I tried each of those as extensively as I was able, and confident that I can't get Ctrl combinations to work, though Left/Right/Up/Down generally do

How can I use Caps as Ctrl for most combinations, but program it to invoke other key presses for some combinations?

Answer 1

Why most of my attempts didn't work
I have figured out that the reason most of these bindings don't work is due to the fact that if you are using Hyper (for example) as your modifier key, it will still be 'pressed' when the Ctrl combination is sent. So the terminal will actually receive Hyper+Ctrl+c, which will likely be treated differently.

The same goes for any other mapping which has a different modifier for the output. The actual modifier you are pressing will still be active, and most applications will handle the two modifiers together differently than if there was just one.

A fairly close solution
A mostly-successful solution can be achieved by binding Caps to the ISO_Level3_Shift modifier, using xmodmap to change the xkb bindings on that layer for the HNEI keys (or HJKL on QWERTY).

You will need xbindkeys for creating the macro and xdotool for sending the resultant combination you want to invoke.

1. Check the keycode for your Caps key with xev -event keyboard (mine is 66)
2. Create .Xmodmap with the following contents, substitute in your keycodes/symbols where necessary (lines beginning with ! are comments)

! Bind Caps to ISO_Level3_Shift modifier
keycode 66 = ISO_Level3_Shift

!Bind the ISO_Level3_Shift modifier to the arrow keys for each
keysym h = h H NoSymbol NoSymbol Left Left
keysym n = n N NoSymbol NoSymbol Down Down
keysym e = e E NoSymbol NoSymbol Up Up
keysym i = i I NoSymbol NoSymbol Right Right

!Remove whatever X currently considers to be it's 'mod5' modifier
clear mod5

!Set ISO_Level3_Shift to be the 'mod5' modifier
add mod5 = ISO_Level3_Shift

The Arch wiki has a good explainer of each column if you are unsure what they are

3. Execute xmodmap .Xmodmap (ensure this executes after setxkbmap in your startup)

4. Find the modifier state when this key (now Caps) is pressed

• Run xbindkeys --multikey
• Focus the window that appears and press Caps + some letter
• You should see a like something like m:0x80 + c; m:0x80 is the modifier state this key generates

5. Use .xbindkeysrc to invoke xdotool on your desired combinations

"xdotool key --delay 0 ctrl+c"
  m:0x80 + c + Release

• Repeat as necessary for other keys
• Make sure to add + Release to the end; I don't entirely understand this but it doesn't work without

6. Make sure that these keys output their regular keycode for their ISO_Level3_Shift layer. This involves editing your .Xmodmap file again.

keysym c = c C NoSymbol NoSymbol c c

• Again repeat as necessary for each of your bindings

• Make sure to restart xbindkeys and run xmodmap .Xmodmap after editing these files to have the changes take effect

Limitations of this solution

• There is a short delay between the key being pressed and the resultant mapping being sent
• The mapping won't repeat when held down, I haven't figured out how to do this

If anybody knows a better solution, or how I can improve this solution, I would love to know! This is cumbersome and imperfect, but is the only thing I've been able to get working so far.