Source

reddish / reddish / public / robots.txt

Full commit
# 80legs
User-agent: 008
Disallow: /

# MSNBot
User-Agent: msnbot
Crawl-Delay: 20

User-Agent: *
Disallow: /*.json
Disallow: /*.rss
Disallow: /*.xml
Disallow: /*?offset=
Disallow: /api/
Disallow: /captcha/
Disallow: /g/*/submit
Disallow: /login
Disallow: /message/
Disallow: /submit