AI Access Audit
How crawler policy and access infrastructure affect retrieval of your website
Declared policy and observed access disagree
The origin-root robots policy permits some discovery identities and disallows others. Separately, Digital Dominator’s identified audit crawler received an edge challenge. The report does not attribute that response to OpenAI, Anthropic, Perplexity or any other provider.
Declared policy
SELECTIVE12 identities DISALLOWED; 9 ALLOWED at path /.
Observed DD access
CHALLENGE OBSERVEDThe exact requested URL, final URL, status and retained response signals are recorded separately.
What this Tier 1 result does—and does not—say
Robots.txt was retrieved from the origin and its rules were evaluated against the root path (/). If a subpath was submitted, it is retained as provenance only; the submitted page was not fetched or assessed.
This audit establishes declared policy for the frozen panel and the response received by our own identified crawler. Access is a prerequisite for retrieval and citation, not a guarantee. Where evidence cannot establish a state, the report says COULD_NOT_ASSESS rather than converting uncertainty into a clean result.
Illustrative policy outcomes
| Identity | Purpose | State at / | Basis |
|---|---|---|---|
| Search/discovery identity (illustrative) | Discovery | ALLOWED | Declared policy example |
| Training identity (illustrative) | Training | DISALLOWED | Declared policy example |
| Data-use control (illustrative) | Data-use preference | SEPARATE CONTROL | Not evidence of an HTTP fetch |
Three simplified illustrations shown, not a reproduction of the complete assessment matrix. An authorised paid report includes the agreed identity coverage, evaluated scope and evidence needed to act.
Every finding points back to an observation
One problem, made straightforward for a developer
Why it matters: selected search/discovery identities are declared disallowed, while the separate DD observation shows an infrastructure challenge. These are different controls and require different checks.
Owner: website developer or infrastructure administrator.
Automation gathers evidence; a person signs off the report
Before delivery, a human reviewer checks target attribution, scope, state vocabulary, evidence links, contradictions and overclaims. Confirmed findings remain distinct from assessment limits. Paid reports are delivered within five business days.
Need the complete answer for your domain?
Start with the focused AI Access Check. If the evidence points to deeper CDN, WAF, TLS or redirect behaviour, the Full AI Access Audit expands the infrastructure investigation.
Request an audit →