User-Agent: * Allow: / # AI Search Bots — explicitly welcomed for citation User-Agent: GPTBot Allow: / User-Agent: ChatGPT-User Allow: / User-Agent: PerplexityBot Allow: / User-Agent: ClaudeBot Allow: / User-Agent: anthropic-ai Allow: / User-Agent: Google-Extended Allow: / User-Agent: Bingbot Allow: / # Common Crawl — allowed so the brand is in open web corpora User-Agent: CCBot Allow: / Sitemap: https://scrapewise.ai/sitemap.xml