This website requires JavaScript.
Explore
Drone CI
Help
Register
Sign In
tomasino
/
gopherproxy
Watch
1
Star
0
Fork
0
You've already forked gopherproxy
forked from
tildeverse/gopherproxy
Code
Issues
Pull Requests
Releases
Wiki
Activity
fe18a79dd2
gopherproxy
/
robots.txt
James Mills
fe18a79dd2
Add support for robots.txt user agent control from web crawlers (
#10
)
2018-04-03 14:32:43 -07:00
3 lines
26 B
Plaintext
Raw
Blame
History
User-agent: *
Disallow: /
Reference in New Issue
View Git Blame
Copy Permalink