How exactly do you overlay an icon? Like a shortcut icon on Windows?
For example, Dropbox overlayed the icons on the files and folders with a tick sign, indicating that they are in sync with the Dropbox servers.
For Microsoft Windows, MSDN describes the process of doing this here.
I found that part of the implementation is in the registry, but it takes much more than just creating registry keys and values to have your own similar overlay implementation. Dropbox may be actively determining what overlay icons to display while it is continually running in the background, using techniques similar to the Code Project linked by Paul (at least in Windows version of Dropbox).
If you're interested in removing the green check-marks (you only want other status icons like blue sync), you can find solutions for doing it in Windows, Mac and Linux here. It is from here that I partly found out how Dropbox is implementing the overlays.
Dropbox lets you use custom status icons for Dropbox.
Here's for Mac:
Copy .ICNS icons to Dropbox contents in Applications.
– Go to Macintosh, then Applications and right click on the Dropbox app icon;
– Select 'Show Package Contents'. When you open Contents, choose Resources folder;
– Copy new (icns) images there and confirm replacement;
– Reboot and launch Dropbox (tested on 10.8.5 and Mavericks)
I put some free icon sets on Gumroad. You'll find a step-by step guide to saving .ICNS files there too.