1. David Soria Parra
  2. hg-website

Commits

David Soria Parra  committed 74b12fc

Add robots.txt

  • Participants
  • Parent commits 4e908cf
  • Branches default

Comments (0)

Files changed (1)

File robots.txt

View file
+User-agent: *
+Crawl-delay: 100
+Disallow: /hg
+Disallow: /repo
+Disallow: /bts/issue?
+
+User-agent: geniebot
+Disallow: /
+
+User-agent: ShopWiki
+Disallow: /
+
+User-agent: DotBot
+Disallow: /