robots.txt — norandesign.com
Last updated: 2026-06-24
-----------------------------------------------
General crawlers
-----------------------------------------------
User-agent: *
Allow: /
Disallow: /zh/ # Prevent duplicate indexing of Chinese pages (use hreflang instead)
Disallow: /api/ # Block any API endpoints if they exist
Disallow: /_framer/ # Framer internal assets
-----------------------------------------------
AI training crawlers — block
-----------------------------------------------
User-agent: GPTBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Claude-Web
Disallow: /
User-agent: Omgilibot
Disallow: /
User-agent: FacebookBot
Disallow: /
-----------------------------------------------
AI indexing/retrieval crawlers — allow
(these surface your content in AI answers, not training)
-----------------------------------------------
User-agent: PerplexityBot
Allow: /
User-agent: YouBot
Allow: /
User-agent: BingBot
Allow: /
-----------------------------------------------
Sitemap
-----------------------------------------------
Sitemap: https://norandesign.com/sitemap.xml