Doesn't Win 98 allow a picture preview in the customise folder options? I forget, as I have been using ME for a long time.
I would say download and install Irfan View. The executable is about 500K so it loads in an instant. You can use it to browse a directory or even make a slide show.
I fine IE annoying for the reason you state. That is one reason why I use Opera which opens multiple windows and has a page bar with tabs to switch quickly between windows. It will also view JPGs, but I rarely use it for that, preferring to browse in Explorer and load image in Irfan View. click here