MADEINRED / METHODOLOGY
AI Search Audit Methodology: Rules and Scoring
Understand the free audit’s crawl access, content discovery and answer structure checks, scoring weights and limitations.
What this audit measures
The audit makes read-only requests to public pages, robots.txt, sitemaps and selected URLs. It applies published technical rules without calling an AI model or changing your website.
This report records technical readiness at the time of inspection and proposes work to do. The score is not a probability of AI citation. Requests using a crawler User-Agent do not prove that an official crawler visited. Indexing, rankings, citations and referral traffic require separate evidence.
How the score works
Crawl access has a 45-point pool, content discovery 25 and answer structure 30. Applicable checks are weighted within each pool. Partial matches receive partial credit; non-applicable checks are excluded. Optional signals can add up to five points, with the final score bounded at 100. Access and indexing problems may cap the total to make important issues visible.
The published caps are 29 for a non-200 public page; 39 for noindex, a robots.txt block on OAI-SearchBot or a restrictive Content-Signal preference; 49 for a blocked OAI-SearchBot test request; and 59 for a robots.txt block on Bingbot. These are MadeInRed’s priorities, not a scoring system from Google or OpenAI.
Business context
Product and purchase signals activate ecommerce checks. Languages, currencies, market references and delivery information can activate cross-market checks. These examine visible facts and discoverability; they do not infer sales, market demand or platform endorsements.
Crawl access · 45 points
| Check | Weight | What to review |
|---|---|---|
OAI-SearchBot access in robots.txtrobots-oai-searchbot | 10 | Review the robots.txt group that applies to OAI-SearchBot and allow the public pages you want search to discover. This is separate from training access. |
ChatGPT-User policy in robots.txtrobots-chatgpt-user | 5 | Review the declared ChatGPT-User policy and test public page access. User-initiated access has different controls from search crawling. |
Bingbot access in robots.txtrobots-bingbot | 5 | Review Bingbot rules for the affected URLs. Allow relevant public content and verify real requests in server logs. |
Googlebot access in robots.txtrobots-googlebot | 2 | Review Googlebot access for the pages intended for Google Search. |
Other AI search crawler policiesrobots-ai-answer-engines | 3 | Review the applicable crawler groups for the answer engines relevant to your audience. |
Content-Signal search and AI-input preferencescontent-signal | 3 | Review search and ai-input declarations in robots.txt or the CDN configuration. Set these preferences to match your publishing policy. |
HTTP access with an OAI-SearchBot User-Agentwaf-oai-searchbot | 9 | Review the recorded HTTP response and CDN rules. Retest the URL with the stated User-Agent and verify actual official crawler visits separately. |
HTTP access with a ChatGPT-User User-Agentwaf-chatgpt-user | 5 | Check the HTTP response, challenges and access rules. A simulated User-Agent is not proof of an official visit. |
HTTP access with a Bingbot User-Agentwaf-bingbot | 4 | Check the HTTP response and any firewall challenge. Use server logs and Bing Webmaster Tools to confirm genuine crawler access. |
Page indexing directivesindexable | 5 | Review robots meta tags and X-Robots-Tag response headers. Remove unintended noindex directives from pages intended for search. |
Snippet permissionssnippet-allowed | 3 | Review nosnippet and max-snippet directives. Allow excerpts where consistent with your publishing policy. |
HTTP and HTTPS healthhttp-health | 3 | Check the final status, redirect chain and response time. Serve the intended public content over HTTPS with a successful status. |
Real 404 responses for missing pagessoft-404 | 2 | Return a real 404 or 410 for missing content and verify an invented URL does not return an ordinary successful page. |
GPTBot training policyrobots-gptbot | Info | Choose your training policy independently of search access. This informational check does not measure citations. |
CDN and firewall identificationwaf-detected | Info | Use the detected provider as a starting point for reviewing crawler access. Provider detection alone does not identify a blocking rule. |
Content discovery · 25 points
| Check | Weight | What to review |
|---|---|---|
Sitemap presence and robots.txt declarationsitemap-present | 6 | Publish a valid sitemap and reference its URL in robots.txt. Include the canonical pages you want discovered. |
Sitemap modification datessitemap-freshness | 4 | Use accurate lastmod values for material updates. Avoid changing dates without a corresponding content update. |
Sampled sitemap URL healthsitemap-urls-alive | 5 | Review the sampled URLs. Replace dead links and redirect entries with their final canonical URLs. |
Public Bing Webmaster Tools verification signalsbing-verification | 5 | Confirm property ownership in Bing Webmaster Tools. DNS verification may not be visible in public HTML. |
Public Search Console verification signalsgoogle-verification | 1 | Confirm property ownership in Search Console. Absence of a public verification tag does not rule out DNS verification. |
Canonical URL consistencycanonical | 3 | Declare a stable canonical URL and align internal links, sitemap entries and redirects with it. |
Language alternativeshreflang | 2 | For multilingual content, declare valid reciprocal hreflang links and correct language URLs. Single-language sites may not need this. |
Cross-market buying informationexport-market-signals | 3 | State languages, target markets, currencies, delivery areas and lead times. Provide accessible company, contact and policy pages. |
Discoverable RSS or Atom feedfeed | 1 | If you publish regularly, provide a valid feed and expose its discovery link in the page head. |
HTTP to HTTPS redirecthttps-redirect | 2 | Redirect HTTP requests to the intended HTTPS URL without unnecessary hops. |
Page title and meta descriptiontitle-description | 2 | Write a specific title and a useful standalone description that match the page’s primary purpose. |
IndexNow submission readinessindexnow | Info | Review your IndexNow key and submission records in your own configuration. Public inspection cannot confirm a private submission history. |
Answer structure · 30 points
| Check | Weight | What to review |
|---|---|---|
Readable content in the initial HTMLssr-content | 6 | Include the main answer and essential facts in the HTML response so they are available without running page scripts. |
Heading hierarchyheading-structure | 3 | Use a clear primary heading and descriptive section headings. Review skipped levels and unrelated sections. |
Headings aligned with reader questionsquestion-headings | 4 | Use headings that help readers locate their questions and the relevant answers. Question phrasing is optional. |
Direct answer passagesdirect-answers | 4 | Place a useful answer near the relevant heading. Passage length is a local heuristic, not an official citation rule. |
Content structure on sampled pagessampled-pages-shape | 5 | Review the sampled product, article or category pages individually. Check readable answers, headings and factual completeness. |
Product and Offer structured dataproduct-schema | 3 | Keep Product and Offer facts consistent with visible product information, including price, currency, availability and applicable identifiers. |
Product and purchase factscommerce-content | 2 | State the product, price and currency, availability, purchase action, and delivery or returns information. Keep these facts visible and consistent with JSON-LD. |
Accessible company, contact and policy pagestrust-pages | 2 | Provide public company, contact and relevant policy pages and link to them clearly. Verify their final HTTP responses. |
Machine-readable publication and update datesdates | 3 | State accurate dates where relevant and keep visible dates consistent with structured data. |
Author or publisher attributionauthor | 2 | Identify the responsible author or publisher in visible content and appropriate structured data. |
JSON-LD structured datastructured-data | 3 | Use valid structured data that describes the actual page. Do not add facts that are absent from visible content. |
Organization identity and sameAs linksorg-entity | 2 | Identify your organization consistently and link genuine official profiles with appropriate sameAs values. |
Useful lists and tablesextractable-structures | 2 | Organise steps and comparisons into semantic lists or tables where they improve the answer. |
Links to supporting sourcesoutbound-citations | 2 | Link factual claims to relevant, verifiable sources. Explain the scope of data and observations. |
HTML language declarationlang-declared | 1 | Declare the page’s actual language in the html lang attribute. |
Open Graph preview metadataopen-graph | 1 | Provide a suitable og:title, og:description and og:image consistent with the page. |
Main content landmarksmain-landmark | 1 | Use main or article elements to distinguish substantive content from navigation. |
Image alternative textimage-alt | 1 | Add descriptive alt text to informative images. Decorative images may have an empty alt attribute. |
Additional signals · +5 points
| Check | Weight | What to review |
|---|---|---|
llms.txt site guidellms-txt | 2 | Consider a concise machine-readable guide to important public content. This is optional and is not a citation requirement. |
llms-full.txt content guidellms-full-txt | 1 | Consider a fuller text guide if useful to your audience. This optional convention is not a ranking guarantee. |
Markdown content negotiationmarkdown-negotiation | 1.5 | If supported, return a faithful Markdown representation for Accept: text/markdown while keeping the same access rules. |
Machine-readable API catalogueapi-catalog | 0.5 | Document public API endpoints in a catalogue if you provide an API. |
Review and AggregateRating datareview-schema | 1 | Use review or rating markup only for genuine, visible reviews and ratings that meet the applicable requirements. |
FAQPage or HowTo datafaq-schema | 1 | Use structured data only when it accurately reflects visible questions, answers or instructions. This does not guarantee a rich result or AI citation. |
Official social and community sourcessource-ecosystem | 1 | Link active, relevant official video and community profiles. A profile link does not measure a citation or prove an endorsement. |
Conditional HTTP response headersconditional-headers | 0.5 | Use ETag or Last-Modified appropriately to support efficient conditional requests. |
Official references
- OpenAI crawler documentation: search, user-initiated access and training are distinct.
- Google crawler documentation.
- Bing Webmaster Tools documentation.
- Schema.org: vocabulary used for structured data.
Page content may quote its source language. Observations are limited to the time and URLs inspected. No score promises indexing, citation or ranking.