<<< Will this work, >>>
<<< or do I need a modem/router? >>>
The problem is that the ISP only allocates one IP address and every computer on the internet needs to have a unique IP address. Doing what you suggest means that only the first PC to connect will get the single IP address and have internet access.
When you install a router, the router gets the single IP address from the ISP and it in turn allocates individual IP addresses to the networked computers using one of the address ranges reserved for use with Local Area Networks, most commonly one of the 192.168.x.x subnets. The router then "routes" the internet traffic to the appropriate computer in the LAN.