Skip to content
Snippets Groups Projects
Verified Commit e24ba55f authored by Thorge Petersen's avatar Thorge Petersen
Browse files

Reordered and consolidated robots.txt entries

parent dd7cf4f5
Branches
Tags
No related merge requests found
User-agent: SemrushBot User-agent: AwarioRssBot
Disallow: / User-agent: AwarioSmartBot
User-agent: PetalBot
Disallow: /
User-agent: DataForSeoBot User-agent: DataForSeoBot
Disallow: / User-agent: PetalBot
User-agent: SemrushBot
User-agent: SEOkicks User-agent: SEOkicks
Disallow: / Disallow: /
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment