After Your Export Site Launches, Will ChatGPT Recommend You? A Build Handover Checklist and InsightWonder
Overseas buyers increasingly ask ChatGPT, Perplexity and Gemini directly. This guide covers the four crawl gates, structured data and llms.txt a build team should leave in place, and how to measure recommendations, citations and competitor share with InsightWonder, including its pricing and limits.
- An AI crawler must pass robots.txt, the WAF, rendering and meta directives before a site can be recommended.
- Allowing a crawler in robots.txt is not enough if a CDN firewall serves it a 403 or a challenge page.
- Google says AI Overviews need no special schema or AI-specific file, so structured data is not a switch.
- InsightWonder counts only citations marked in the answer and never shows unmeasured results as 0%.
- InsightWonder does not measure Chinese engines or Google AI Overviews; use DaleGEO for mainland China.
When an export brand launches an independent store or an overseas corporate site, the build team usually hands over an SEO checklist: titles, descriptions, a sitemap, page speed, mobile layout. All of that still matters. But overseas buyers now have another way to shortlist suppliers. Instead of opening a search engine, they ask ChatGPT, Perplexity or Gemini which vendors are reliable for a product category, or how two options compare. The brands named in that answer, and the pages it cites, are often where the buyer's shortlist begins.
This article covers two things. First, what a build team should leave in place at handover so that AI systems can at least read the site. Second, once the site is live, how to use an AI visibility tool such as InsightWonder to measure whether the brand is recommended, which sources get cited, and how much of the conversation competitors own.
Disclosure: InsightWonder and RelayWonder are run by the same team as this site; this article is based on each product's public pages, and prices and features may change.

Why launch day is no longer the finish line
Classic search returns a page of links and leaves the comparison to the buyer. An AI assistant returns a finished answer that names brands, gives reasons and sometimes links to sources. For an exporter, ranking reasonably well in search does not mean appearing in AI answers, and the reverse is also true. Researchers call optimization for these generative engines GEO. The GEO paper by Aggarwal et al. introduced the term and, on its own benchmark, found that changing how content is written could raise a source's visibility in generative responses by up to 40%, with results varying by domain.
That paper points in a direction rather than offering a guarantee. It shows that how content is written affects whether a generative engine picks it, but only if the engine can get to the content in the first place. Many independent stores fail exactly there: the pages look fine in a browser, yet an AI crawler receives nothing useful. So before anyone talks about being recommended, the build team has to make being readable part of the delivery.
Four gates a crawler has to pass
Break down how an AI system reads a site and you find at least four gates that must be passed in sequence. Fail any one and the quality of the content behind it is irrelevant. The InsightWonder AI crawlability check works through the same four: robots.txt, real crawler identity (user agent plus WAF), JavaScript shells versus rendered content, and meta directives. It also fetches the page as an ordinary browser for comparison, and states that a 200 response with an empty body is reported as a failure, not a pass.

Gate one: robots.txt
robots.txt is the first file a polite crawler reads. Google's robots.txt guide describes it as a way to manage crawler traffic by telling crawlers which URLs they may access. It is not a way to hide pages, and it cannot enforce behavior: reputable crawlers follow it, others may not.
AI companies usually run more than one crawler, each with a different job, so they need separate rules. Take OpenAI. Its crawler documentation lists three user agents. OAI-SearchBot surfaces sites in ChatGPT's search features. GPTBot collects content that may be used to train foundation models. ChatGPT-User covers visits triggered by a user inside ChatGPT, is not used for automatic crawling, and robots.txt rules may not apply to it. The documentation is explicit that these settings are independent: you can allow OAI-SearchBot and block GPTBot, staying visible in search answers while keeping content out of training.
The usual failure at handover is a broad Disallow line left by a theme or security plugin, or a blanket rule that blocks every user agent containing the word bot to stop scraping. Go through these rules before launch, decide explicitly for each AI crawler, and record the decision in the handover document so the client can revisit it. InsightWonder keeps an AI crawler list that is handy for checking names and purposes.
Gate two: crawler identity and the WAF
This is the gate people miss most often. Many independent stores sit behind a CDN with a firewall, anti-scraping rules or bot management that judge requests by user agent, IP reputation or rate. The result is a robots.txt that says allow, while the real crawler reaches the edge and gets a 403, a CAPTCHA or a blank challenge page. A browser loads the site normally, so neither the agency nor the client notices.
OpenAI's documentation also publishes IP ranges for each crawler and recommends allowing them so legitimate traffic reaches the site. In practice, build WAF allow rules around verified crawler identity rather than the user agent string alone, which anyone can spoof. Then request key pages with the real user agent and confirm that the response contains content rather than a challenge.
Gate three: rendered content, not a JavaScript shell
Many modern front-end frameworks render in the browser by default. The server returns a nearly empty HTML shell and JavaScript paints the content afterwards. Google's JavaScript SEO basics explains that Googlebot queues pages for rendering in headless Chromium, yet still recommends server-side rendering or pre-rendering, partly because not all bots run JavaScript. With AI crawlers it is safer to be conservative: if the product description, specifications and company information are not in the raw HTML, assume they are not being read.
Key pages on an export site, meaning the homepage, product and category pages, the about page and the FAQ, should ship their content through server-side rendering or static generation. The test is simple: disable JavaScript, or request the page from the command line, and check whether the product name and key paragraphs are in the returned HTML.
Gate four: meta directives
The last gate is the robots meta tag and the X-Robots-Tag response header. Staging environments often carry noindex, and it survives into production. Some front ends output noindex and then remove it with JavaScript. The same Google guide warns that when a crawler sees noindex it may skip rendering, so removing the tag with script may not work; if you want a page indexed, do not put noindex in the original code. For AI crawlers with weaker rendering, that rule only gets stricter.
As a pre-launch checklist, the four gates look like this:
- robots.txt: record an allow or block decision for each major AI crawler, such as OAI-SearchBot and GPTBot, keeping search crawlers and training crawlers separate.
- WAF and CDN: write rules using each vendor's published IP ranges or official verification method, then retest key pages with the real user agent to rule out 403s and challenge pages.
- Rendering: the raw HTML of key pages must contain the body content without relying on the browser to run scripts.
- Meta directives: confirm that no noindex or nosnippet is left in production and that no X-Robots-Tag header is misconfigured.
- Retest: run all four gates again after every redesign, theme change or CDN rule change.
Structured data and llms.txt: useful, not a ticket
The four gates make a site readable; structured data makes it understandable. Schema.org is a shared vocabulary founded by Google, Microsoft, Yahoo and Yandex that lets sites mark up organizations, products, FAQs and more in a consistent way, so machines do not have to guess. At minimum, an export site should ship Organization markup for the company, Product on product pages and FAQPage on the FAQ, with the markup matching what is visible on the page.
Be honest with clients about what it does. Google's page on AI features says plainly that there are no additional requirements to appear in AI Overviews or AI Mode, no need for new AI text files, and no special schema.org markup to add. The page has to be indexed and eligible to show with a snippet. Structured data is a good habit for clarity, not a switch.
The same goes for llms.txt. According to llmstxt.org, it is a Markdown file at the site root with an H1 for the site name (the only required element), a blockquote summary, and H2 sections listing links to important pages, so an AI agent can grasp the site quickly. It is still a proposal. Adding one at handover costs little, but do not promise a client that it will get them recommended. InsightWonder's llms.txt guide goes into the format in more detail.
After launch: measure with InsightWonder
With the technical groundwork in place, the client's real question is whether the brand shows up when overseas buyers ask AI. That needs a measurement tool. InsightWonder is an AI visibility and GEO platform for brands and growth teams. It measures whether AI answer engines name, recommend and cite a brand for the questions buyers ask, and traces each result back to the original answer and the cited URL. It covers ChatGPT, Gemini, Perplexity, Claude and Grok. The interface is English-first, with a Simplified Chinese version for teams who prefer it.
Start with the free tools, which need no account, and use them to repeat the checks above:
- AI visibility snapshot: enter a site and run 5 questions against 1 model for a first impression.
- AI crawler checker: fetch pages as AI crawlers to catch robots.txt and firewall blocks.
- Page extractability check: see how much body text remains when scripts are not run.
- llms.txt validator and schema extractor: confirm the file is well formed and the markup is actually output on the page.
A free snapshot tells you whether there is a problem. To follow trends you need the paid workspace, which runs in three steps: build knowledge, measure, then create and distribute. The modules most useful for export sites are:
- Buyer questions: full-sentence questions across research, evaluation, comparison, trust and purchase intents. Generated questions must not contain your brand or a competitor's name, and the tracked set stays fixed across runs so results remain comparable.
- AI visibility: mention rate is the primary metric, shown as a question by model matrix. Questions containing your brand are excluded from scoring, unmeasured cells are never shown as 0%, per-engine results come before averages, and each run records its methodology version.
- Citation sources: only sources marked inside the answer text count as citations, not the retrieval pool, and cited versus recommended are tracked separately.
- Competitor intelligence: counts every brand an answer names, not only the list you enter, merges spelling variants, and withholds share of voice when the sample is thin.
- Content opportunities: flags an opportunity only where a competitor appeared and you did not, without claiming causation.
Two lessons it publishes are worth remembering. First, it says that counting the retrieval pool once inflated its own citation figures by two to four times, which is why it now counts only sources marked in the answer. Second, it observed a site's mention rate move from 3% to 13% after web search was enabled, while the site itself changed nothing. A single reading depends on measurement conditions, so trends are more reliable than any one number. Its Q3 2026 AI recommendation benchmark ran 240 brand-free questions across 5 models; according to the report, Perplexity cited sources in every answer and reddit.com was cited in all 12 categories, which is useful context on how different engines cite.
Pricing and which plan to pick

According to the InsightWonder pricing page, billed monthly in US dollars:
- Starter: $39, 1 site, 30 tracked questions, 750 AI answer runs per month.
- Growth: $149, marked most popular, 1 site, 100 questions, 2,500 runs.
- Scale: $389, 3 sites, 300 questions, 7,500 runs.
- Plus: $899, 5 sites, 500 questions, 12,500 runs.
- Enterprise: custom pricing.
One question sent to one model equals one answer run, so 30 questions across five engines is 150 runs per round. Annual billing offers two months free, team seats are unlimited, and media placement is pay-per-use through a wallet. There is no permanent free plan; eligible new accounts get a trial with terms shown at signup. For a single-brand export site, a sensible order is to run Starter for a month, see where competitors appear and you do not, and only then decide whether to track more questions.
What it does not do
Recommending a tool means stating its limits. Based on InsightWonder's own pages, including its comparison with Profound:
- No Chinese engines: DeepSeek, Doubao and other mainland assistants are not measured. Brands selling into mainland China should use DaleGEO, which measures those engines.
- No Google AI Overviews or AI Mode, and no Microsoft Copilot.
- Measurement runs through model APIs, not the consumer apps. Its privacy page says it currently uses Perplexity's API, which serves the GPT, Claude, Gemini and Grok families, so results may differ from what a user sees in an app.
- Its customers page has no published customer case yet, and the homepage dashboard numbers are sample data, not results.
- No ranking guarantee and no automatic publishing: distribution is draft-first and every external write is logged.
- No public API or multi-client workspace, and the default cadence is monthly (configurable), not daily.
- The Shopify app is labelled as launching with the platform, so do not treat it as available in the App Store yet.
None of this undermines its use as a post-launch health check. But if a client's main market is mainland China, or what they care about most is Google's AI summaries, they need a different tool or a combination.
Building this into the delivery
For teams that build cross-border independent stores, AI visibility does not need to be a separate project. It fits into delivery milestones that already exist:
- Development: render key pages on the server or generate them statically, output structured data from the templates, and control staging noindex with an environment variable so it never reaches production.
- Before launch: run the four-gate checklist, recheck crawler access, extractable text, llms.txt and schema with the free tools, and write the robots.txt and WAF decisions into the handover document.
- First month after launch: agree a set of brand-free buyer questions with the client, run a baseline, and record mentions and citations per engine.
- Ongoing: review trends monthly, fill content gaps where competitors appear and the brand does not, and keep every piece draft-first with human sign-off before publishing.
The client then receives more than a site that is live. They get a technical baseline they can recheck and a fixed question set they can track over time. If the project also involves content strategy and ongoing optimization, see our GEO service. One last point: when and how much a brand gets recommended depends on the industry, competitors and the content itself, usually over months, and no tool or agency should promise rankings.
FAQ
Should an export site block GPTBot in robots.txt?
It depends on whether the client is comfortable with their content being used for model training. OpenAI's crawler settings are independent: GPTBot relates to training, while OAI-SearchBot decides whether the site can appear in ChatGPT search results. A common setup is to allow OAI-SearchBot, decide on GPTBot with the client, and then confirm the WAF is not blocking the crawlers you allowed.
Can InsightWonder measure DeepSeek or Doubao?
No. By its own description, InsightWonder covers ChatGPT, Gemini, Perplexity, Claude and Grok. It does not cover mainland Chinese assistants, Google AI Overviews or Microsoft Copilot. Brands targeting mainland China should use DaleGEO, which measures engines such as Doubao, DeepSeek and Qwen.
Will adding llms.txt and structured data get my site recommended by AI?
Not on their own. llms.txt is still a proposal, and Google states that appearing in its AI features requires no special AI file or schema. Both make a site easier to understand, but whether it gets recommended depends on the content, the competition and the engine, so measure continuously after launch instead of treating them as a one-time fix.
Sources
- Overview of OpenAI Crawlers · OpenAI
- Robots.txt Introduction and Guide · Google Search Central
- Understand JavaScript SEO basics · Google Search Central
- AI features and your website · Google Search Central
- About Schema.org · Schema.org
- The /llms.txt file · llmstxt.org
- GEO: Generative Engine Optimization · arXiv (Aggarwal et al., KDD 2024)