Hi, I have uploaded an xml sitemap to Google and they say my robots file is 404 error, I do not have a robots file as I want no exclusions, how do I write a file that syas this? Also how long do you think it takes to get indexed in Google?Thanks
I suggest that you contact Google and ask them what they need. I would expect it to be a text file on your website called "robots.txt" which contains:
User-agent: * Allow:
I don't want Google, or any other search engine to list my website (it is only there for me to experiment with html, css, php and other coding). Googlebots visit my site regularly but only look at my "robots.txt" file, which says "Disallow". This stops them looking any further.