Unlike the Xbox 360 wireless controller, which required a separate dongle for PC use, all you need to use the Xbox One controller on PC is a standard Micro-USB cable. The wired connection provides the power, so you don't even need batteries. Unfortunately, you can't use the Xbox One controller wirelessly on a PC at this time.
To set up the controller, head to Major Nelson's blog and download the drivers for either a 32-bit (x86) or 64-bit (x64) machine. (If you're unsure which file to grab, open Control Panel, type About in the search bar and click System. Then, look under 'System Type.')
Open the file you just downloaded. Amusingly enough, Windows will show an "unknown publisher" warning, even though the drivers are coming from Microsoft. Click Run in the box that pops up. You don't have to plug in the controller to install the drivers.
A setup wizard should appear. Read and accept the license agreement, wait for the installation to finish, then click Finish.
Now, plug in your Xbox One controller. It may vibrate briefly, and you should see a "driver software installed successfully" message on the System Tray.
At this point, you should be ready to play. Games that support Microsoft's Xbox 360 controller should automatically recognize the Xbox One controller without any additional setup (with some exceptions described below).
The biggest problem is that several of the games I tested would not recognize the controller, including Dark Souls II, Transistor and Eldritch. The controller did recognize several other games, including Dishonored, Super Meat Boy and Trials Evolution. Uninstalling and reinstalling the drivers did not help, so hopefully this is just a temporary issue that Microsoft and game developers can resolve through updates.
Also, the first time I installed the driver, Windows 7 showed a "Runtime Error" after plugging in the controller, followed by an "XboxStat.exe has stopped working" error. The controller still worked, but the message was a nuisance. These messages went away after reinstalling the driver.
Finally, keep in mind that plugging the controller into a PC will break its wireless connection to the Xbox One. You'll have to plug the controller back into the console via USB to pair them again.