I am navigating with CMD+TAB through the app-switcher, but if I release the keys on a closed window, I get only the focus on that application. Is there a way to reopen that window again, with a shortcut?
If you hold option before releasing command, the application switcher will reopen a default window if there's no open windows.
Witch has an option to make that the default behavior:
The shortcuts for opening a new default window depend on the application, but some common ones are ⌘N, ⌘1, ⌥⌘1, and ⌘0.
Many apps follow an unofficial convention of using Cmd-1 for the primary window. For everything else, you can use the custom shortcuts available in the Keyboard pane of system preferences to define a custom shortcut for any menubar action, including the one typically under the Window menu used to recall the primary window (it's name varies with every application).
I do this myself for every app that doesn't have a shortcut key preset.