Surprisingly I found that IE9+ has this built-in:
In Firefox you can simply Paste & Go in the address bar (don't think there's any shortcut assigned to it).
I'm sure there are plenty of extensions for FF, Chrome and other browsers to open URLs directly from the clipboard.
If you want to stick with desktop shortcuts, here's something that might help. Get NirCmd, then modify your shortcuts such that they launch NirCmd first and it in turn opens the specified browser with the URL from the clipboard:
As the names state the shortcut on the left will launch IE with the URL currently in the clipboard, and the one on the right will do the same with FF. These are generic shortcuts that don't need to be modified every time.
The full Target lines from the screenshot are:
I:\nircmd.exe exec max "C:\Program Files (x86)\Internet Explorer\iexplore.exe" ~$clipboard$
I:\nircmd.exe exec max "C:\Program Files (x86)\Mozilla Firefox\firefox.exe" ~$clipboard$
Modify as required, obviously. See the NirCmd help file for many more useful commands, variable names (such as "~$clipboard$") and the like. You could, for example, create a batch file that uses NirCmd to read a list of URLs from a text file and copy each one to the clipboard, or launch different browsers with each one and so on.