XP: Plug in the computer to the router with ethernet cable. Right click the icon in the taskbar, select status. Click support tab. The routers address will be the default gateway.
Vista: Use a cable again. Open network and sharing centre. Manage network connections. Right click the connection, select status. Click details, the address will be listed.
Hope this helps!