tiny little gray box set top right of the window.
Not a bug but a feature! (not joking)
This is a Firefox icon to list all opened tabs... You can removed it this way:
Firefox orange icon -> Options -> Personalize tool bar... just drag and drop the icon.
EDIT : to complete my answer and avoid misunderstanding ;) :
Firefox troubleshooting tips:
1) Try in Firefox safe mode: may be the problem comes from an add-on or a plug-in (some plug-ins are not installed in the Linux Firefox version...)
If the problem disappear you near to find the culprit: a plugin or add-on
2) Try with another Firefox Profile
If the problem disappear this way just create a new profile and transfers your old profile informations to the new (check in the Ff help)
3) Try with another Windows user account
If the problem disappear at this stage, there's some setup or parameter of your profile in cause. (A security utility or some "eyes candy setup" for example...)
4) Try in Windows safe mode
If the problem disappear here, it's probably not an hardware problem (beware: this is not always true: say at 75%...)
How to delete a Firefox plug-in (not only disable) if needed.
In Firefox about:config set the plugin.expose_full_path parameter to TRUE
In Firefox in about:plugins you'll have the full address of the installed plugins
If a disabled plugin (in safe mode or otherwise) is suspected it's possible to delete the plugin (if needed) by deleting the file related to this plug-in (a "dll" for example...).
Hope this help...