Yes you need to configure the settings for the router to give it the user name and password for your internet access.
Did the router come with a set-up disk or wizard? This should guide you through the router set-up. You can do it directly by typing in the appropriate address for the router in your browser's address line. Try http:// 192..168.2.1 (without the space) but if you tell us the model number of your Belkin router more help can be given.
It is by far the best to set up the router with it connected to your PC with a network cable rather than wireless. When everything is okay then you disconnect the cable and use the wireless adaptor.