The Belkin router's IP address is 192.168.2.1 and typing this into your web browser should open its configuration pages. You do *NOT* need to be connected to the internet to do this.
If it doesn't open the pages, it may be because your "Local Area Connection" in the PC hasn't got its IP address automatically from the router. You then need to give it an IP address manually.
In the Network Connections folder, right click the Local Area Connection and select Properties. Highlight the entry for "Internet protocol (TCP/IP)" and click the Properties button. In the Properties sheet that opens, click "Use the following IP address and enter these details:
IP address - 192.168.2.5
Subnet Mask - 255.255.255.0
Default Gateway - 192.168.2.1 (the *ROUTER'S* IP address)
DNS Server - 192.168.2.1 (the *ROUTER'S* IP address)
Click OK to close all boxes and see if you can then access the router's configuration pages.
Once you have the router working, ideally you should go back into the TCP/IP settings and set *BOTH* IP and DNS to get their addresses automatically.