Ahrefs domain authority integration
The readiness report now includes domain-level authority signals from Ahrefs alongside the existing technical checks. Adds critical context on why a technically-perfect site might still struggle with AI citation.
Added
- Domain Rating (DR) via Ahrefs API, color-coded by tier
- Referring domains count (live)
- Live backlinks count
- Ranking keywords count
- Secondary metrics: estimated organic traffic, Ahrefs Rank, all-time backlinks
- Ahrefs metrics now included in the downloadable PDF report
- 24-hour server-side cache per domain so repeat scans return instantly
Changed
- PageSpeed Insights API call now goes through our
/api/pagespeed.php proxy with caching
- Eliminated anonymous-quota rate limiting (we have dedicated PSI API key now)
AI Instructions page detection
Added detection for the emerging AI Instructions page pattern. Audits whether the scanned domain publishes a dedicated citation-prompt page at common URL patterns.
Added
- Probes
/ai-instructions/, /ai-info/, /ai/, /ai-instructions.md, /ai-information/
- Reports which path was found, plus context on why this pattern matters
- Counted as a Discoverability frontier standard
Frontier agentic standards section
New dedicated section probing 23 emerging standards for the agentic web. Marked as "Bonus" signals so early adopters get recognition without penalizing typical sites that don't support them yet.
Added (23 new checks)
- Discoverability: Link header for llms.txt, DNS for AI Discovery (DNS-AID), security.txt, humans.txt, RSS/Atom auto-discovery
- Content Accessibility: Markdown content negotiation (
Accept: text/markdown)
- Bot Access Control: Content Signals (
noai, noimageai, TDM-Reservation), Web Bot Auth (HTTP Message Signatures directory)
- Protocol Discovery: NLWeb (Microsoft),
ai-plugin.json, OIDC discovery, DID configuration, MCP Server Card, Agent Skills, WebMCP, API Catalog (RFC 9727), OAuth discovery (RFC 8414), OAuth Protected Resource (RFC 9728)
- Commerce: x402 Payment Required, Merchant Payments Protocol (MPP), Unified Commerce Protocol (UCP), Agent Commerce Protocol (ACP)
Changed
- Fetch proxy now exposes HTTP response headers via
X-Proxy-Headers for header-aware checks
- Added support for custom
Accept header (used by markdown content negotiation check)
- New
/api/dns-txt.php endpoint for DNS-AID record lookups
Smart sitemap discovery
The tool now finds sitemaps in 7 common locations and reads the Sitemap: directive in robots.txt before falling back. Fixed the case where WordPress and Yoast sites were incorrectly flagged as having no sitemap.
Changed
- Parses
Sitemap: directives from robots.txt first
- Falls back to:
/sitemap.xml, /sitemap_index.xml (WordPress + Yoast), /wp-sitemap.xml (WordPress core 5.5+), /sitemap-index.xml, /sitemap1.xml, /sitemap/sitemap.xml, /sitemap.php
- Validates response contains
<urlset> or <sitemapindex> before counting as found
- Reports the actual location found in the result detail
Fixed
- WordPress + Yoast sites were incorrectly failing the sitemap check despite having
/sitemap_index.xml
Tool relaunch as Website AI Agent Readiness Check
Major rebrand and URL change. Expanded scope, deeper schema coverage, action bar with PDF / Compare / Share, and Google Lighthouse integration.
Changed
- Renamed from "Agent Readiness Checker" to "Website AI Agent Readiness Check"
- Moved from
/tools/agent-readiness/ to /tools/website-ai-agent-readiness-check/ (with 301 redirect)
- Hero copy updated to mention 35+ signals (now 50+)
Added
- Google Lighthouse integration (Performance, Accessibility, Best Practices, SEO) via PageSpeed Insights API
- Compare to competitor mode: paste a second URL, see side-by-side category-level scores
- Download PDF report button (multi-page branded PDF via jsPDF)
- Share this report button (Web Share API + clipboard fallback)
- 7 new high-value schema-type checks: Organization, WebSite, SearchAction, Article-with-attribution, FAQPage, BreadcrumbList
- TTFB measurement from proxy timing
- Redirect chain length measurement
- Mobile viewport meta check
- HSTS header check
- Character encoding check
- hreflang language alternates check
Cosmetic and UX improvements
Added
- Live action bar with PDF, Compare, Share buttons
- Score-circle animated ring on render
- Category bars now animate from 0 to final value on reveal
Changed
- Improved mobile layout for the action bar (now wraps cleanly)
- Fixed contrast of category-bar labels on small screens
v1.0
May 18, 2026
Initial
Initial release
First public version of the Agent Readiness Checker. 17 checks across 5 categories, scored as a 0-100 readiness grade with category breakdown and per-check fix recommendations.
Initial checks
- AI Crawler Access: robots.txt rules for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, Bytespider, Meta-ExternalAgent, CCBot
- Discoverability: HTTPS, robots.txt, sitemap.xml, llms.txt, llms-full.txt
- Structured Meaning: Page title, meta description, canonical URL, lang attribute, OpenGraph, JSON-LD schema, heading hierarchy, semantic HTML, image alt text
- Rendering & Content: Server-side rendering verification, content depth (word count), internal link density
- Trust Signals: Author attribution
- 0-100 scoring with category breakdown and letter grade (A through F)
- Per-check pass/warn/fail with fix recommendations