FrontPage 2002 and robots.txt

  Simon_P 20:46 22 Sep 04
Locked

Using FP 2002’s feature of listing the top 10 visited pages the robots.txt comes up in the top 10 visited pages. Is there a way to prevent this from being shown? (Probably not without removing the file?) I assume that this is because the googlebot etc read this file frequently.

Not really a major problem just one of those niggling things.

Also just about in the next few days, going to upgrade to FP 2003, and would the same problem exist?

  Simon_P 22:56 22 Sep 04

It's not really a problem, but it is visible in the top ten pages that all can see, not that it really matters I suppose, as no real harm can be done if people see it. The top ten pages is not really an important page, it just gives visitors an idea of what is most popular.

Its a good point about using it to calculate real visits, thanks.

I guess it does go to show how many times the bots hit sites (lots)

Just one of the silly niggles, I guess I should just leave it alone and follow my own rule "if it aint broke dont fix it!"

  Simon_P 17:56 23 Sep 04

It was just one of those things that I put on in the beginning when I started learning, and was reluctant to remove it for no real important reason. I guess I just didn’t want to just remove it altogether. Putting on a top ten manually is a good idea.

Thank you for your time and help

Simon

This thread is now locked and can not be replied to.

How to get Windows 10 for free | How to install Windows 10: There is still a way to avoid paying…

1995-2015: How technology has changed the world in 20 years

Alex Chinneck’s giant ice cube Christmas tree at Kings Cross

Apple rumours & predictions 2017: The iPhone 8, new iPads, and everything else you should expect fr7…