I don't know about IE but in Firefox, there is a 'square' icon for new window. Mine is between the Forward/Backward arrows and the Refresh icon, which are located just below the Main Menu at the top left of the window.
It may be that I put it there so, if you don't have it, right click on the tool bar, alongside the Menu, click on Customise, look for the icon in the window, then click and drag into position. You only get the 'hand' to move icons while this window is open.
Hope that helps you?