I find the best way to do it is to copy paste (for files, i don't know if it works for playlists.
On XP, go to my documents>my music>itunes and copy that folder to an external source (unless you changed it or do not set itunes to copy files when adding to library all your songs should be in there also) and then when you re-install copy back into the same location. Open itunes and go to file>add folder to library and select the itunes music folder within the itunes music folder (confusing I know) and it should bring back in all songs with artwork. Same for Vista except the folder is in <username> > music > itunes. If your songs are not stored in this folder, copy it anyway and paste it back into the same location when you have reinstalled, then follow the above instructions to reimport and it should detect all your album artwork. I don't know about WMP11.
Another important thing, if you have bought any music from the itunes store make sure you go to the store menu and select deauthourize computer before you reinstall, otherwise itunes will detect your re-install as a new pc and it will use up one of your five allowed computer credits.