WebRipper is a powerful tool which can search any given URL for the file types you specify, automatically downloading any matching files it finds.
You might have found a site with a lot of interesting PDF's, for instance. Normally you'd have to browse each page in turn, looking for links, and clicking each one individually. But if you point WebRipper at the page then it can search for and download every document in a fraction of the time.
To try this out you must create a download job, which tells WebRipper exactly what you want to do.
This starts by specifying the source URL(s) and the file types of interest. But you can also provide a user name and password, if the site requires it, and set the minimum and maximum size of the files to be downloaded.
But most important is the "Link restrictions" option, which defines where WebRipper will search. By default this is set to "unrestricted", meaning the program will follow links, even if they're to other websites. This could result in you trying to download a huge number of files, though, so we'd recommend you choose the "within domain only" option, which keeps the program focused on the initial site.
WebRipper won't work with all websites. And while it ran just fine when we tried it, the program's last version came out in 2009 and it's now unsupported.
Still, it's also powerful, free, and has no adware or other hassles. Take a look.