RecomazeBot
Web Crawler for AI Readiness Audits
RecomazeBot is the web crawler used by Recomaze AI Readiness Audit to analyze websites and evaluate their optimization for AI assistants like ChatGPT, Perplexity, and Google AI.
User Agent
What RecomazeBot Does
- ✓Analyzes publicly accessible pages for AI optimization factors
- ✓Checks schema markup, structured data, and metadata
- ✓Evaluates content quality and AI readability
- ✓Measures page performance and technical SEO factors
- ✓Checks robots.txt compliance for AI crawlers
Crawl Behavior
Frequency
On-demand only. We crawl when a user requests an audit of their website.
Pages per visit
Limited to 10-20 pages per audit to minimize server load.
Rate limiting
Respectful delays between requests. We don't overload servers.
Robots.txt
We respect robots.txt directives for RecomazeBot.
What We Don't Do
- • We don't store or republish your content
- • We don't access password-protected areas
- • We don't submit forms or modify anything on your site
- • We don't crawl continuously or index your entire site
- • We don't share crawl data with third parties
How to Block RecomazeBot
If you don't want RecomazeBot to crawl your website, add this to your robots.txt file:
User-agent: RecomazeBot Disallow: /
Note: If you block RecomazeBot, users won't be able to run AI readiness audits on your website through our service.
Questions or Concerns?
If you have any questions about RecomazeBot or notice any issues, please contact us at: better@recomaze.ai