I highly doubt such an extension already exists. One possible alternative could be an app like ThumbsPlus:
ThumbsPlus is a highly customizable image database with thumbnails and batch editing. It makes it easy to catalog, organize, locate and maintain all of your graphics, multimedia and font files. It runs on Windows XP, 2003, 2008, Vista and Windows 7 and 8. Both 32 and 64-bit operating systems versions are supported.
Apparently it has just the feature you seek:
The thumbnail generated for WAV files is simply the icon of the application associated with them in Windows Explorer. To generate thumbnails from rendered audio data, see: Options | File Loading - Movie / Sound, Render audio waveforms option.
Render audio waveforms: Select this option to have ThumbsPlus render audio data using the selected style. Available styles are Wave, Radial, Diagram, Tunnel , and Frequency.