llm: GPTBot
policy: allow
comment: OpenAI's GPTBot is allowed to access content for research and model training.

llm: Google-Extended
policy: allow
comment: Google AI is allowed to access content for AI training purposes.

llm: ClaudeBot
policy: allow
comment: Anthropic's ClaudeBot is permitted to access content.

llm: CCBot
policy: allow
comment: Common Crawl is allowed to access content for general indexing.

llm: PerplexityBot
policy: allow

llm: Amazonbot
policy: allow

llm: You.com
policy: allow

llm: FacebookBot
policy: allow

llm: Bytespider
policy: allow

llm: cohere-ai
policy: allow

llm: gpt-collect
policy: allow

llm: DuckDuckBot
policy: allow

llm: ai-crawler
policy: allow

llm: yandexbot
policy: allow

llm: Sogou
policy: allow