<<< How do I configure the router to do this >>>
Don't configure the router, although you could disable DHCP.
Give the PCs the fixed IP addresses you want them to have. In the Network Connections folder, right click the Local Area Connection (or Wireless Network Connection) and select Properties. Highlight the "Internet Protocol (TCP/IP)" entry and click the Properties button. Click "Use the following IP address" and enter:
IP address - 192.168.1.100 (or as appropriate)
Subnet Mask - 255.255.255.0
Default Gateway - 192.168.1.1 (the router's IP address)
DNS Server - 192.168.1.1 (the router's IP address)