Remote Desktop Connection on Windows (mstsc) saves the previous computers you have connected to in the "Computer" dropdown.
How can I clear them, or selectively delete entries?
If you navigate to the following location in the registry, you should be able to see a list of connections made;
"HKEY_CURRENT_USER\Software\Microsoft\Terminal Server Client\Default"
Under this key you should be able to remove any entries.