tildeverse
/
tilde.news
mirror of https://github.com/tildeverse/lobsters
5
3
Fork 3
Code Issues 2 Releases Wiki Activity

block archivers

This commit is contained in:
joshua stein 2012-10-24 11:53:21 -05:00
parent 5ee6c227a3
commit dc71bbbd98
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
public/robots.txt
View File

@ -0,0 +1,5 @@
# no caching
User-agent: ia_archiver
Disallow: /
User-agent: archive.org_bot
Disallow: /