The best I can think if is forcing the browser to open in a specific mode by using a command-line option. Sadly, anything like this cannot be found from the "official" list of command-line options for IE. Also discussions on Technet also seem to imply that - at least on IE9 - you cannot permanently set document mode to a legacy setting.
Also I gotta say it puzzles me why you need to figure this out on client-side in the first place. I mean if you want to force IE10 users to use IE9 rendering mode then you just need to add an
X-UA-Compatible meta tag with IE version at 9:
<meta http-equiv="X-UA-Compatible" content="IE=9">
From MSDN Library:
In certain cases, it might be necessary to restrict the display of a
webpage to a document mode supported by an earlier version of Internet
Explorer. You can do this by serving the page with an x-ua-compatible
header. For more info, see Specifying legacy document modes.