I just installed Firefox 3.6 and am also seeing this behavior. I actually hadn't noticed until you brought it up, to be honest.
So what we're going to do is work with the registry, a la the instructions provided here. Essentially we're going to force HTML files to display an appropriate file type icon. I will edit this answer with more details in a few minutes after testing the technique.
UPDATE
First off, that technique should work, and the keys you should be looking for are HKEY_ROOT\.htm, HKEY_ROOT\.html and HKEY_ROOT\FirefoxHTML.
However, I'm noticing that I already had HKEY_ROOT\FirefoxHTML\DefaultIcon configured, and even after configuring HKEY_ROOT\.htm and HKEY_ROOT\.html as well (all to use Firefox's executable icons), and rebooting, and manually rebuilding the icon cache, I still have blank icons.
I'm not forced to believe that the issue is with the Firefox executable itself. However, you may have better luck specifying a .ico file.