There are a number of useful websites on home networks. For example:
In short, you need network cards in each machine connected using straight network cables with rj45 plugs (unless you're going for a wireless setup).
(I think) You usually configure the router by accessing a website. This info will be set out in the manual. I hope this helps.