detect-ai-agent-traffic-openclaw

The visits your site actually saw, with the agent rows flagged as the ones your logs miss. The essence of the post before any header appears.

Playbook

Detecting agent traffic

AI agents are already reading your site.
Here is how to see them.

Default logs drop the one header that gives them away

Visits, this morning

What your site actually saw

Chrome on macOS/pricingBrowser

OpenClaw/docs/getting-startedAI agent · not in your logs

Firefox on iPhone/Browser

Claude Code/blog/geo-auditAI agent · not in your logs

Cursor/docs/apiAI agent · not in your logs

Agents leave one signal a browser never sends.Accept: text/markdown

A person asks for the page, an agent asks for the text. Keeps the one signal and the 80% token fact, without a raw header up front.

Playbook

Detecting agent traffic

Agents ask for your site differently.
That is how you spot them.

Token counts from Cloudflare, measured on its own blog

A person

“Give me the page.”

Navigation, footer, scripts, styles: the whole thing, 16,180 tokens of it.

Accept: text/html

An AI agent

“Just give me the text.”

Markdown, 3,150 tokens. Agents ask this way because it is 80% cheaper to read.

Accept: text/markdown

No person ever asks for markdown. That one word is the signal, and 15 lines of middleware log it.

One large card per agent the method detects: OpenClaw, Claude Code, Cursor, OpenCode, each with its real mark. The registered cover.

Playbook

Detecting agent traffic

How to identify AI agents
hitting your site.

Works on Vercel, Cloudflare, AWS CloudFront and NginxCheckly study of 7 agents

OpenClaw

Accept: text/markdown, text/html;q=0.9, */*;q=0.1

Claude Code

Accept: text/markdown, text/html, */*

Cursor

Accept: text/markdown,text/html;q=0.9,application/xhtml+xml;q=0.8,…

OpenCode

Accept: text/markdown;q=1.0, text/x-markdown;q=0.9, text/plain;q=0.8,…

Four different headers, one word in common. No browser ever sends it, and 15 lines of middleware log it.