gitea: dis-allow a few AI crawlers that cause too much load
This commit is contained in:
parent
0d30074d8b
commit
ce850f7210
@ -2,3 +2,18 @@ User-agent: *
|
||||
Disallow: /*/*/archive/*.bundle$
|
||||
Disallow: /*/*/archive/*.tar.gz$
|
||||
Disallow: /*/*/archive/*.zip$
|
||||
|
||||
User-agent: Amazonbot
|
||||
Disallow: /
|
||||
|
||||
User-agent: Bytespider
|
||||
Disallow: /
|
||||
|
||||
User-agent: ClaudeBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: DataForSeoBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: FacebookBot
|
||||
Disallow: /
|
||||
|
Loading…
x
Reference in New Issue
Block a user