How much of the internet is bots? AI agent traffic in 2026

On September 30, Cloudflare said that this year, for the first time, more than half of internet traffic wasn't human. Cloudflare sits in front of more than 20% of the web, so it sees a large slice of what moves online.

The numbers behind it are large. At the end of 2024, Cloudflare handled an average of 63 million requests a second. Now it's 115 million, with peaks above 150 million. Over the same year, daily requests from AI agents on its network grew by more than 1,700%.

For a growth marketer, the useful question is which of those visitors are working for a customer, and whether your website lets them in.

Three things worth knowing

What percentage of internet traffic is bots?

About half, and a bit more each year. Thales, the company behind the Imperva Bad Bot Report, found that bots made up 53% of web traffic in 2025, and people 47%.

The shift is slow. Bots were 47.4% of traffic in 2022, 49.6% in 2023 and 51% in 2024. Thales called 2024 the first time in a decade that automated traffic passed human traffic.

Before you quote these numbers, keep two things in mind.

• "Bots" is a wide group. It counts Google's search crawler, AI tools and attack bots. Thales classed 40% of that traffic as malicious, so attack bots make up a large part of it.
• Each company measures its own network. Thales counts traffic across its customers. Cloudflare counts its own network. The figures land close together, but they aren't one global census.

Is the dead internet theory true?

Partly. The dead internet theory says the internet is now mostly bot activity and automated content, and that real people have been crowded out. The first half matches the data: most requests are automated.

The second half doesn't hold in Cloudflare's numbers. Cloudflare says the human web didn't shrink to make room. A second audience arrived alongside it.

It feels true for some sites, though. Cloudflare reports that in some of the most heavily crawled categories, including retail, computer software, IT services and financial services, human traffic fell by as much as 40% in under a year. AI answer engines read the page and give the person a summary, so the bot visits and the human doesn't.

If you run one of those sites, your server is busier while your visitor numbers drop. That's the version of the dead internet that matters to a growth team.

What is agentic web traffic?

Agentic traffic is software visiting websites on behalf of a person. Someone asks ChatGPT to compare three project tools, and ChatGPT fetches the pages to answer. That fetch is agent traffic.

Cloudflare describes agents as sitting between humans and traditional bots. They don't look at ads, but there's usually a person behind them with a job to get done. Their traffic even follows human habits, with a weekly rhythm and a dip over the summer holidays.

There are now three kinds of automated visitor worth telling apart.

Training crawlers are growing fast too. In spring 2025, 22% of the crawler requests Cloudflare saw were for AI training. By June 2026 it was 52%.

The difference that matters for marketing: a training crawler collects your page once to build a model. An agent comes back each time someone asks about that topic. Agent traffic grows with the number of questions people ask, not with how much you publish.

Why does AI agent traffic matter for growth marketers?

Because some of that traffic is a customer. Cloudflare's examples are an agent booking a table, comparing insurance quotes or buying a dataset for a researcher. In Cloudflare's words, turn an agent away and you may be turning away the person who sent it.

Take a buyer who asks an AI assistant for the best accounting tool for a 10-person team. The assistant reads a handful of pages and names three products. If your pricing page blocked the agent, the answer is built from your competitors' pages and whatever else it could reach.

That's why AI visibility is becoming its own number. It tracks how often AI answers name your brand for the questions your buyers ask. What an AI visibility score really tells you explains how to read it, and where it can mislead.

How do you block AI bots on Cloudflare without losing search traffic?

Block AI training, not search. In July 2026, Cloudflare replaced its single "block AI bots" switch with three separate controls: Search, Agent and Training. They're available on every plan, including the free one, so you can leave Search and Agents on Allow and limit only Training.

The data showed why one switch wasn't enough. Fewer than 1% of sites on Cloudflare block search crawlers, while 17% block training. Site owners want to be found. They don't want to give their content away for nothing.

The hard part is crawlers that do two jobs. Googlebot, Bingbot and Applebot collect pages for search and for AI. If you choose Block, you also drop out of their search results. On September 15, Cloudflare added a setting called Disallow AI Training. It keeps you in search while telling those crawlers not to train on your pages. Apple, Google and Microsoft have committed to honor it, though Bing's support through robots.txt is only planned for early 2027.

For new domains, Cloudflare now suggests settings based on how the site makes money. Ad-funded sites can block agents on pages that carry ads, because an ad only pays when a person sees it. A SaaS or B2B site usually wants the opposite: agents reading its product, pricing and comparison pages.

How do you track AI crawler and agent traffic in Cloudflare?

If your website runs through Cloudflare, a report called AI Crawl Control shows how many AI bots visit, which company sends them and why they come. It's included in the free plan. On the free plan, Cloudflare recognizes AI bots by the name they send with each request. Paid plans add Cloudflare's more thorough bot detection.

The numbers below are example data for a mid-sized SaaS site, so you can see what each view looks like at real volume. The menu names match the Cloudflare dashboard.

Step 1: open Cloudflare AI Crawl Control

Log in at dash.cloudflare.com, pick your domain and choose AI Crawl Control in the left menu. The Overview page gives you the headline numbers: total requests from AI bots, the change against the previous period, the most crawled page, and the bots grouped by company.

In our example, Cloudflare counts about 41,500 successful requests from AI bots in a week. Google sends the most, followed by OpenAI, Anthropic and Perplexity.

Step 2: sort the traffic by purpose

Go to Metrics. In the Requests over time chart, change Group by from Crawler to Category. This is the view that answers the question most people have: how much of my AI traffic is someone asking a question right now, and how much is a company collecting pages for its models?

Cloudflare sorts AI bots into these groups. The names in the dashboard aren't obvious, so here is what each one means.

AI Assistant is the number to watch, and one of the most useful agentic metrics you can track. ChatGPT-User, Claude-User and Perplexity-User visit only when a person asks a question and the assistant decides to read your page to answer it. When that count rises month over month, your pages are being pulled into more answers, which usually means AI tools are naming your brand more often. Treat it as an early signal, not proof: an assistant can read a page and still leave you out of the answer, so compare it with how often the answers actually name you.

Step 3: find GPTBot, ClaudeBot and ChatGPT-User visits

Go to the Security tab in AI Crawl Control. It lists every AI bot with two numbers: Allowed (it got the page) and Unsuccessful (it was blocked or hit an error).

In the example week, OAI-SearchBot (ChatGPT search) got 6,240 pages, GPTBot (training) 3,720 and ChatGPT-User 2,410. ChatGPT-User is the closest thing to counting real people asking ChatGPT about your topic, because it only visits when someone asks.

Look at the Unsuccessful column too. In the example, Claude-User got 690 pages and failed 1,240 times, and Perplexity-User got 410 and failed 820 times. Cloudflare says unsuccessful requests can come from any rule or error, not only a block you set, so a high number is a reason to check your firewall and bot settings. If they are turning away assistants, they are turning away the people behind them.

Step 4: find the pages AI crawlers request most

Back in Metrics, scroll to Most crawled paths. It ranks your pages by how often AI bots requested them. The Patterns tab groups them into sections, such as everything under /blog/.

The pages near the top are the ones shaping what an AI tells a buyer about you. In the example, that's pricing and a comparison article, which matches what buyers ask: what does it cost, and how does it compare? Keep the facts on those pages current and in plain text.

On paid plans, the same page also shows referrals: visits from people who clicked through from chatgpt.com, perplexity.ai and other AI services. On the free plan, look for those sites in the referral report of your analytics tool.

Step 5: choose which AI bots to allow

The Search, Training and Agents controls sit under Security, then Settings. Filter by Bot traffic and open AI Crawler and Agent access policies.

We allow all three, because we want to be read, cited and recommended. If you'd rather keep your content out of AI training, change only Training, and use Disallow AI Training for the crawlers that also run search. To block one specific bot, use its toggle in the AI Crawl Control Security tab instead.

Every chart has a Download CSV button, so you can save a weekly snapshot and compare months later.

What is Web Bot Auth, and how do sites spot fake AI agents?

Web Bot Auth lets AI companies sign their agents' requests so a website can check they're genuine. It matters because anyone can write "ChatGPT" in a request and pretend to be OpenAI. Two new approaches make faking harder.

Web Bot Auth lets the company running an agent sign each request, so a website can check it really came from them. OpenAI, Google and AWS sign their agents' requests this way, and Cloudflare sees more than 500 billion verified bot requests a week.

Precursor, a Cloudflare tool launched July 13, 2026 as part of its paid Enterprise Bot Management, watches how a visitor behaves across a whole visit: how the mouse moves, how long a click takes. People move in arcs and make small corrections. Scripts tend to move in straight lines and click with a precision no hand can match. In one 24-hour period it ran 206 million checks across 73,438 sites.

Cloudflare has also found that one visit can switch between a person and an agent. Someone browses a store, then hands the checkout to a shopping assistant. A website that blocks every agent breaks that sale.

What is x402, and can AI agents pay for content?

x402 is an open standard for payments over the web, and AI agents are starting to use it to pay for content. When an agent asks for something that costs money, the website answers with a "402 Payment Required" message. The agent pays and asks again. No account or form is needed.

Cloudflare uses x402 in Monetization Gateway, a closed beta for eligible US customers that lets a site charge per request, per question or per piece of text. A second beta, Pay Per Use, pays publishers when an AI company actually uses their content, not when it crawls. Both are early, and Cloudflare says pricing and discovery aren't solved yet.

Most growth teams won't need agent payments this quarter, but they show where things are going: agents are being treated as customers, and access is becoming something websites set terms for.

How do you start measuring AI visibility?

Start with five checks. Most take minutes.

The bottom line

Bots passing people online sounds like the dead internet theory coming true. Look closer and a large part of the new traffic is people, working through software. The websites that do well will tell those visitors apart: welcome the agents that bring customers, set terms for the crawlers that only take, and measure whether AI answers send buyers their way.