The magnifier allows a user to invert the colors of the entire Desktop.
Does anyone know how to invert the colors of just an application, or a specific window?
There are only three modes that Magnifier works in, and the high contrast colour inversion only works in conjunction with these modes. These modes are listed in the Microsoft page Make items on the screen appear bigger (Magnifier):
So to achieve what you require, you will have to use in Docked mode, specifying a region occupied by a window (adjust the window to your particular tastes beforehand). Unfortunately you can not thereafter move the window and expect the magnifier to follow it, but you can always "re-dock" the magnifier to the new window position. Not ideal, but better than nothing.
This is the manner in which I use it. Hope this helps.