Email: page 1

• Email me when the file changes January 25, 2017

  It is important to ensure that Google does not index sites whilst they are still on a staging environment, but you cannot lock it down completely - how would your clients proof it? So I run a simple global rewrite rule in Apache that redirects all requests for robots.txt to a central disallow all response. This works great and Google appears to honour the rule as one would hope. What happens though when something about that central file changes? Read more ⇒