Your problem could be related to your IE addons or also if you have a long restricted sites list (tools -> security -> restricted sites):
Check Microsoft's How to make IE open new tabs faster:
Add-ons are loaded by IE when you open a new browser window or tab. This is usually a quick process, but certain add-ons may cause IE to take a longer time than expected. For example, after installing Skype 4.1, this user (and his father) encountered a slowdown in IE on their computers.
Examine the performance impact of your add-ons using the Manage Add-ons dialog. Click on Tools - Manage Add-ons, and look at the ‘Load time’ column.
If you do not want to keep a slow loading add-on around, you can disable it, and it won’t be loaded the next time you open a new window or tab. You can also open Add/Remove Programs and remove IE add-ons that you are not using. This removes the add-on for all users on the computer.
If your problem is not related to your addons, then use this file (from Adding unwanted sites to the Internet Explorer Restricted Zone) to delete all your restricted sites:
http://www.mvps.org/winhelp2002/DelDomains.inf (right-click -> install)