I would guess that when going either through DNS or via IP address there is a difference
in the identification of the computer that is originating the RDP request.
The difference might be in the user-name acquiring a different qualifier,
for example WORKGROUP.
I suggest to logon using both methods, then in a Command Prompt (cmd)
use the whoami command to find the exact user account that is used:
whoami /user
If there is a difference, then you have found your answer.
[EDIT]
As you have found that in both cases there is exactly the same user in the same domain
and with the same SID,
then my only explanation is a difference that is not visible to you.
It might be that because of the DNS request your connection request does not follow the
exact path in the network that is taken when using the IP address.
My personal conclusion is that Windows in that case does not search for the connection
in the right place, does not find the existing connection and so opens up a new session/desktop.
However, when the login process executes in this new desktop, it logs you in under
your correct user account / SID, so the same user ends up as having two desktops.
You might be able to further research that possibility using the LogonSessions
utility from Sysinternals, which gives more detailed information about
the currently active logon sessions. Another possibly useful utility is EnumWinsta GUI
which displays the list of window stations and desktops.
More information can be found in the article :
Windows Sysinternals : Windows Core Concepts - Sessions, Window Stations, Desktops, and Window Messages.
In other words, I believe this is an undocumented bug in RDP, which you should report.
Unfortunately, Microsoft Connect does not accept bugs for Windows,
so your only option is to go on the Microsoft Windows Forums
and hope that someone from Microsoft will pass it on to be corrected in some unknown
time in the future.