So, GIMP can't do that alone - what is needed is: a program that is associated with XCF files and gerenerate thumbnails.
The specfications for it are at:
(Thanks to the answer here: https://stackoverflow.com/questions/9572669/how-can-i-create-thumbnail-preview-icons-in-windows-explorer-for-document-files)
And them, you don't need to have this program to actively read and render XCF files: GIMP adheres to the Free Desktop standard for thumbnails, as detailed here:
given the image path as an URI of the kind "file:///c:/my_directory/subdir/image.xcf", take the hex-form of the md5 hash of this path, and save the thumbnail as a png file named
"<md5>.png" in a
.cache folder on the user home directory.
So, if one can build a minimal program that requires as a Windows "active icon" extension that just reads the thumbnails on the correct directory (and resizes them as required by the Windows API parameters) that program would work.
As listed in the other answers, there might be bigger programs that already do that. But if someone reading here is interested in contributing to GIMP for Windows eco-system, and find himself capable of creating such a program, it would be a nice addition to the GIMP-Windows project. (maintained by volunteers who build and create te installers of GIMP for Windows - the project tself has lttle code of its own - mostly for the Windows installer)