Don't remove IE/Outlook, etc. Use Avant Browser instead & configure it to be your default browser - click here. It's freeware, no ad/spyware & has a lot of great features. IMHO a must try, & check out all it's features closely. I've tried a few other browsers, incl. Mozilla but Avant remains the best (for me, anyway).
Sorry, don't know enough to comment about the security holes in IE, etc.