While SageThumbs or XnView work well, they are overkill (in terms of disk space, memory, and CPU usage) for just showing .hdr file thumbnails. For that, you only need one small (56 kB) DLL from Microsoft:
Download the (enormous, half-gigabyte) DirectX SDK from Microsoft.
http://www.microsoft.com/en-us/download/details.aspx?id=6812
Run the installer and install only the Utilities. (This requires the Runtime to be installed, too.)
Copy the file TxView.dll from the Utilities\bin\x64 directory in the DX SDK directory into \Windows\System32\.
- If using 32-bit windows, use the
x32 directory instead.
From the command-line:
cd \windows\system32
regsvr32.exe TxView.dll
That's it! You can now delete the remaining 142MB of Utilities installed by the SDK. You don't even need to restart Windows Explorer, just F5 in a folder with .hdr files and you'll see thumbnails for them!