Have you installed Window Live Mail? You can access all your accounts that are linked from there. So you can also remove an account. To get the Mail app back, click on the Store tile, once opened then right click and select 'Your Apps' and Mail will be under 'Apps not installed on this PC'.
I use Windows Live Mail from the desktop mainly. It's the upgrade from the old 'Outlook'.