Agent Analytics
From crawler request to conversion — the complete chain.
Agent Analytics evaluates your server logs and identifies every AI crawler: GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot, Google-Extended, and others. You see which pages are fetched how often, which responses fail (404, 403, timeouts), and whether the activity ultimately results in Citations, referral visits, and Conversions.
What you gain
- Complete overview of all AI crawlers on your domain — separated by training and live-search bots
- Page-Level: Which URLs models actually read and which they never reach
- Make errors visible: 404, 403, timeouts and robots.txt blockades that make you invisible
- Attribution: AI-Referral-Traffic and Conversions per engine instead of flying blind in your analytics tool
In three steps
- 1Connect logs or tracking
Connect server logs, CDN logs, or the tracking snippet. Bot signatures and IP ranges are automatically verified to prevent fake user agents from being counted.
- 2Evaluate crawl behavior
For each bot, you see frequency, page coverage, status codes, and response times — including pages that are never fetched despite high priority.
- 3Measure impact
Crawl data is linked with Citations and Referral-Traffic. This way, you can identify where the chain breaks and which page needs work next.
This is what Agent Analytics looks like
Time range and engine selection apply to both views and update them together.
Where does the crawler data come from?
From your server or CDN logs, or the tracking snippet. Bots are identified via User-Agent and verified IP ranges to prevent fake accesses from being included in the analysis.
Why does a bot crawl a lot, but I'm not cited?
Crawling only means content has been fetched. Whether it is cited as a source depends on structure, topicality, and authority — the funnel makes precisely this gap visible.
Can I exclude individual bots?
Yes. Agent Analytics shows you the impact: which Citations and Referral-Traffic are tied to a bot before you block it via robots.txt.
How is AI traffic separated from normal traffic?
Via referrer domains of the engines and optional UTM parameters. Visits without a referrer can additionally be attributed using time windows and landing page patterns.