You are probably talking about Utility Windows or Panels.
From Apple's Human Interface Guidelines:
(all boldface highlighting by me)
Panels are either application-specific or systemwide. Application-specific panels disappear when the application is deactivated.
A user can open several panels at a time; they float on top of document windows. When a user makes a document active, all of the application’s panels should be brought to the front, regardless of which document was active when the user opened the panel. When your application is inactive, its panels should be hidden. Panels should not be listed in the Window menu as documents, but you may put commands to show or hide all panels in the Window menu. Figure 14-38 shows examples of different styles of panels.
This article explains who's resonsible for their behavior:
Mac OS X utility windows differ from their Windows counterparts in several significant ways:
- They have a drag strip that enables users to reposition them on the desktop, but the drag strip usually does not have a text title.
- When your application is in the background (that is, its windows are inactive but visible), its utility windows should be hidden from view. (You are responsible for implementing this behavior.)
- In addition to application-specific utility windows, Mac OS X also supports system-wide utility windows, which remain visible even when the parent application is hidden from view.
Highlighting again by me.
Seeing as application developers need to implement this behavior themselves, I don't see a sane way around this behavior (sane meaning, neither changing the applications' code at runtime, nor changing the Cocoa frameworks somehow). You can, of course, petition the Transmission folks to use a regular window without this behavior (like changing from Finder's Show Inspector to Get Info).
If I'd have to guess, you want to do something unusual, such as create software documentation. That's probably why this behavior doesn't work for you.
As a workaround, quickly make screenshots of these panels and view them while working in another application.