robots.txt — norandesign.com

Last updated: 2026-06-24

-----------------------------------------------

General crawlers

-----------------------------------------------

User-agent: *
Allow: /

Disallow: /zh/ # Prevent duplicate indexing of Chinese pages (use hreflang instead)
Disallow: /api/ # Block any API endpoints if they exist
Disallow: /_framer/ # Framer internal assets

-----------------------------------------------

AI training crawlers — block

-----------------------------------------------

User-agent: GPTBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: Omgilibot
Disallow: /

User-agent: FacebookBot
Disallow: /

-----------------------------------------------

AI indexing/retrieval crawlers — allow

(these surface your content in AI answers, not training)

-----------------------------------------------

User-agent: PerplexityBot
Allow: /

User-agent: YouBot
Allow: /

User-agent: BingBot
Allow: /

-----------------------------------------------

Sitemap

-----------------------------------------------

Sitemap: https://norandesign.com/sitemap.xml