mirror of
https://github.com/tildeverse/lobsters
synced 2024-06-20 23:47:04 +00:00
block robots from /search
This commit is contained in:
parent
4b768e2f42
commit
ce8eb45008
|
@ -14,3 +14,6 @@ Disallow:
|
||||||
|
|
||||||
User-agent: Baiduspider
|
User-agent: Baiduspider
|
||||||
Disallow: /
|
Disallow: /
|
||||||
|
|
||||||
|
User-agent: *
|
||||||
|
Disallow: /search
|
||||||
|
|
Loading…
Reference in New Issue
Block a user