short and simple:
what is a GTK engine vs a theme etc?
In GTK, an engine changes the look of the widgets drawn, often to match the native widget set on a particular OS, to control properties such as widget curvature or animations. An engine is software running on your machine.
Themes are a general term for all of the artwork & the color schemes used when rendering everything you see. It's not a GTK-unique term. Two different engines could use the same theme files as input, but the final result would be rendered a bit differently. Themes are a (mostly) static collection of files.
Both themes and engines can be used to make your programs look different. A theme is simpler than an engine. It typically just involves creating a collection of image files that get used to construct window borders and widgets. An engine is like a plugin to a web browser that actually runs new code to generate the borders and widgets. Engines are more complex to create, but this allows them to create fancier effects.
The theme engine is the display backend that uses a theme to style windows.