Whilst I was designing the humber mouth website, I did not want anybody viewing it until it was complete. To stop search engines picking up the site on their searches I used what is called a robot exclusion standard. It is a piece of code which sits at the root of the site in a txt file.
The code included in robot.txt:
User-agent: *
Disallow: /
* = select all search engines
/ = disallows access to all parts of the site
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment