Can AI crawlers read levels.io?
AI crawlers can find and read this site well. 4 of 20 checks failed.
Scored 85/100 on 2026-10-14 from https://levels.io/
What an AI crawler gets
The first 400 characters of text in the HTML, before any JavaScript runs.
@levelsio's blog Subscribe Nav: home · stats · projects · rss · api · llms · mcp · investments · contact Sort: latest · oldest · views · blog views · views 30d · blog views 30d Filter: all · blog · X posts · long form · presentations · podcasts Layout: list · cards Hi. I’m @levelsio and you’ve found my blog! I build internet startups by myself: Nomads.com, Remote OK, Hoodmaps, Photo AI and In...
74,688 characters of readable text
Which AI crawlers are allowed
| Crawler | robots.txt says |
|---|---|
| GPTBot | Allowed |
| OAI-SearchBot | Allowed |
| ClaudeBot | Allowed |
| Claude-SearchBot | Allowed |
| PerplexityBot | Allowed |
| Perplexity-User | Allowed |
| Google-Extended | Allowed |
| CCBot | Allowed |
| Bytespider | Allowed |
| Amazonbot | Allowed |
Score by category
Fixes, biggest first
Add structured data (JSON-LD)
Machine-readable facts about your business: name, logo, what you sell. AI systems use it to describe you accurately.
Describe your organisation in JSON-LD
Organization or WebSite objects tie your site to a known entity, so answers name you correctly.
Offer a machine-readable interface
A valid /openapi.json or /llms-full.txt lets AI agents use your product or read your docs in full, not just your homepage.
Mirror llms.txt at /.well-known/llms.txt
Some tools look in the .well-known folder instead. A copy of the same file covers both.
Get told when this changes
We're building monitoring: a re-score every week and an email when the score moves. Leave your email to get it first.
Show your score
Add this badge to your site or README. It always shows the latest score.
16 checks passed
- Publish a robots.txt file (4 points)
- Let GPTBot crawl the site (6 points)
- Let ClaudeBot crawl the site (6 points)
- Let PerplexityBot crawl the site (6 points)
- Add an llms.txt file (3 points)
- Put your words in the HTML (18 points)
- Stop serving an empty app shell (10 points)
- Add real copy to the page (4 points)
- Give the page an h1 heading (4 points)
- Use semantic HTML tags (4 points)
- Write a descriptive page title (3 points)
- Add a meta description (3 points)
- Add Open Graph tags (2 points)
- Add a canonical link (2 points)
- Publish a sitemap.xml (6 points)
- Keep sitemap dates fresh (4 points)