Can AI assistants read gridbloom.app?
An AI-Visibility Snapshot of the founder's own product, run on Oct 20, 2026. Everything below was observed that day.
gridbloom.app/robots.txt
User-Agent: * Allow: /
Lets every crawler in
Blocked request
403 blocked
- When
- October 20, 2026, 8:40 pm Central
- Crawler
- ClaudeBot
- Page
- gridbloom.app/
- Response
- "Your request was blocked."
Readiness checks
Requests named after AI crawlers were turned away
- Blocked
Crawler access
Requests named after AI crawlers were blocked or challenged. robots.txt invites them in.
- Partly
Technical and structured data
Organization and WebPage data are present. The sitemap is challenged.
- Clear
Entity clarity
The title and description say what Gridbloom does.
- Full audit
Web presence
Brand and category searches are in the Full audit, not the Snapshot.
- Observed:
- we saw it
- Inferred:
- our reading of what we saw
- Illustrative:
- an example, not a measurement
What a crawler gets back
~ % curl -si -A "ClaudeBot/1.0" https://gridbloom.app/
HTTP/2 403
Your request was blocked.
~ % curl -s https://gridbloom.app/robots.txt
User-Agent: *
Allow: /
…
Sitemap: https://gridbloom.app/sitemap.xml
~ % curl -si -A "OAI-SearchBot/1.0" https://gridbloom.app/sitemap.xml
HTTP/2 403
cf-mitigated: challenge
<title>Just a moment...</title>
<meta name="robots" content="noindex,nofollow">
Observed. Run from outside the site on October 20, 2026, 8:40 pm Central. Output trimmed to the lines that matter.
Crawler access
Invited in, then turned away.
- Blocked Observed
- GPTBot, ClaudeBot, CCBot: "Your request was blocked."
- Challenged Observed
- Every other crawler name: a check page marked noindex.
- What we can't see Inferred
- Real crawlers come from their companies' own addresses. Cloudflare's security log shows what they got.
Requests to gridbloom.app
One per crawler name, Oct 20, 2026
- GPTBot OpenAIBlocked
- ClaudeBot AnthropicBlocked
- CCBot Common CrawlBlocked
- OAI-SearchBot OpenAIChallenged
- ChatGPT-User OpenAIChallenged
- Claude-SearchBot AnthropicChallenged
- Claude-User AnthropicChallenged
- PerplexityBot PerplexityChallenged
- Perplexity-User PerplexityChallenged
- Googlebot GoogleChallenged
- bingbot MicrosoftChallenged
- Applebot-Extended AppleChallenged
robots.txt
User-Agent: * · Allow: /
7 fixes. The first 3 are here.
Fix firstP0-1Crawler accessSmall effort
AI crawlers are blocked, though robots.txt invites them in
- Observed
- Requests named GPTBot, ClaudeBot and CCBot got "Your request was blocked." robots.txt says "User-Agent: *, Allow: /".
- Inferred
- This matches Cloudflare's setting that blocks AI training crawlers.
- Fix
- Decide what you want. If assistants should learn about Gridbloom, allow these crawlers in Cloudflare. If blocking training is the point, say so in robots.txt too, so both give the same answer.
Fix firstP0-2Crawler accessSmall effort
Search and answer crawlers get a check they can't pass
- Observed
- Requests named OAI-SearchBot, ChatGPT-User, Claude-User and PerplexityBot got a "Just a moment..." page marked noindex. Headless Chrome stalled on it too. A normal Chrome window got through.
- Inferred
- The real crawlers come from their companies' own addresses and may be let through. That can't be seen from outside.
- Fix
- In Cloudflare's security events, filter the last 7 days by these crawler names. If they were challenged, let verified bots skip the challenge.
Fix nextP1-1Technical and structured dataSmall effort
The sitemap is behind the same check
- Observed
- robots.txt points to /sitemap.xml and /community/sitemap.xml. Both returned the challenge page to every request outside a browser.
- Fix
- Exempt robots.txt, the sitemaps and llms.txt from the challenge. Then confirm in Google Search Console that the sitemap was read.
In the full report
4 more fixes and the prompt kit are in the full report, sent by email.
Or check your own site: AI-Visibility Snapshot, $49