Web Scraping Without Getting Blocked in 2026: Proxy and CAPTCHA Benchmark

Getting blocked when web scraping is not a code problem. It's an infrastructure problem. The same Python script that fails every second request on a Cloudflare-protected target will sail through if you swap the proxy tier and add a CAPTCHA solver — no other changes needed.

This post is the infrastructure companion to our WAF bypass benchmark and our Playwright stealth guide. Where those posts cover detection bypass at the browser and fingerprint layer, this one covers the two layers underneath: proxy selection and CAPTCHA solving. We ran proxy type tests across four target tiers and CAPTCHA solver benchmarks across three services in April 2026. Here's what the numbers look like.

The Three Layers That Determine Block Rate

Before choosing tools, understand what's actually blocking you. Most scraper failures come from one of three infrastructure layers — and fixing the wrong one wastes time:

Layer 1 — IP reputation. Your request's ASN (the network block your IP belongs to) is the first signal most WAFs check. Cloud provider ASNs (AWS, GCP, Azure, Hetzner) are pre-flagged. A perfect browser fingerprint on a datacenter IP still gets challenged on Cloudflare Enterprise.

Layer 2 — CAPTCHA challenges. When IP reputation passes but behavior triggers a soft block, the site serves a CAPTCHA. If your scraper can't solve it, the session dies. The gap between CAPTCHA solver services on modern challenge types (Turnstile, reCAPTCHA v3) is large enough to change project viability.

Layer 3 — Request rate and session patterns. Even with good IPs and CAPTCHA solving, fixed-interval requests at high frequency trigger rate limiting. Most sites deploy velocity-based blocking that's entirely separate from fingerprint detection.

Fix the right layer for your target and your block rate drops dramatically. Fix the wrong one and you spend money without moving the needle.

Proxy Type Benchmark: Success Rates by Target Tier

We ran 300 requests per proxy type against four target tiers: generic e-commerce (no WAF), basic Shopify (Shopify Protect), Cloudflare standard (Business tier), and Cloudflare Enterprise. All requests used curl-cffi with Chrome TLS impersonation and randomized timing (300–2,000ms jitter). The only variable was the proxy tier.

Proxy Type	Generic E-com	Shopify Basic	Cloudflare Std	Cloudflare Enterprise
Shared datacenter	74%	61%	19%	4%
Dedicated datacenter	88%	73%	31%	9%
Residential (rotating)	97%	94%	79%	52%
ISP / Static residential	99%	97%	85%	61%

What this means in practice:

Shared datacenter proxies are not viable for Cloudflare-protected targets — a 19% pass rate on standard tier means you're paying for four requests to get one result. Dedicated datacenter improves things but doesn't cross the threshold where scraping becomes economically sensible on protected targets.

Rotating residential proxies are the minimum viable proxy tier for any target behind a major WAF. ISP proxies (residential IPs assigned by ISPs rather than mobile or home broadband pools) deliver the best results per dollar on Cloudflare standard — the 6-percentage-point improvement over rotating residential is consistent across multiple test runs.

The one variable not captured in this table: residential proxy pool quality varies significantly between providers. Premium residential pools (Bright Data, Oxylabs, Decodo) outperform budget providers by 8–15 percentage points on Cloudflare Enterprise specifically, because Enterprise-tier fingerprinting cross-references IPs against known proxy provider ranges.

CAPTCHA Solving Services: Speed, Cost, and Accuracy

CAPTCHA challenges are binary — your session lives or dies on whether the solver returns a valid token before the challenge expires. We benchmarked three services across four CAPTCHA types in April 2026: 2Captcha, CapMonster Cloud, and CapSolver.

Service	reCAPTCHA v2	Cloudflare Turnstile	hCaptcha	Cost per 1K (reCAPTCHA)
2Captcha	10–30s	15–25s	12–22s	$2.99
CapMonster Cloud	18–35s	6.24s	14–28s	$0.60
CapSolver	12–20s	4–8s	10–18s	$0.80

The Turnstile gap matters. Cloudflare Turnstile has replaced reCAPTCHA v2 on a large portion of protected targets as of 2026. CapMonster's 6.24s average solve time on Turnstile is more than 2x faster than 2Captcha, and CapSolver is faster still on average. For projects where Turnstile is the primary challenge type, 2Captcha's higher accuracy on legacy reCAPTCHA v2 (100% success in our tests) is less relevant than its Turnstile lag.

Cost vs accuracy tradeoff: 2Captcha is the most expensive service we tested and the slowest on modern challenge types — but it delivers the most consistent accuracy across reCAPTCHA v2, Invisible reCAPTCHA, and legacy image CAPTCHAs. For targets still using legacy challenge types, the reliability premium is worth it. For Cloudflare Turnstile-heavy targets, CapSolver or CapMonster save cost without sacrificing success rate.

Built-in solvers in scraping frameworks (Playwright-stealth's built-in Turnstile handler, Camoufox's solver bridge) skip the external API round-trip entirely. In our Playwright stealth benchmark, built-in solver integrations reduced average CAPTCHA resolution time by 40% compared to 2Captcha on Turnstile. The tradeoff: built-in solvers require a headless browser runtime, which adds memory overhead and is overkill for targets that don't need browser-level fingerprinting.

Request Frequency: The Block Rate Curve

Rate limiting is the layer most scrapers hit after fixing proxy and CAPTCHA issues. The block rate curve varies by target tier, but the pattern is consistent: block probability stays low up to a threshold, then rises sharply.

Request Rate	Generic E-com	Shopify	Cloudflare Protected
<1 req/sec	<3%	<5%	<8%
1–3 req/sec	5–10%	10–18%	20–35%
3–8 req/sec	15–30%	30–50%	55–75%
8+ req/sec	45–70%	70–85%	85%+

The non-obvious insight: fixed-interval delays are worse than random jitter at the same average rate. A scraper sending 1 request exactly every 2 seconds is more identifiable than one sending requests at 0.8s, 3.1s, 1.4s, 2.7s intervals — even though the average rate is the same. Detection systems flag rhythmic patterns.

Practical implementation: use a random delay between min_delay and max_delay rather than a fixed sleep. For Cloudflare-protected targets, min_delay=0.8s, max_delay=4.0s keeps block rates below 10% at the residential proxy tier. Exponential backoff on 429 responses (1s → 2s → 4s → 8s) prevents session bans on temporary rate limit hits.

For large-scale jobs, distribute requests across multiple sessions rather than increasing single-session rate. Ten sessions at 0.5 req/sec each outperforms one session at 5 req/sec both on block rate and on session longevity.

Choosing Your Stack by Target Type

The right combination of proxy tier, CAPTCHA solver, and request rate depends on your target. Here's the decision matrix we use at ScrapeWise before scoping any new project:

Generic e-commerce (no WAF): Shared datacenter + no CAPTCHA solver + up to 3 req/sec. Block rate under 10%. Cheapest setup; no need to over-engineer.

Shopify or basic WAF: Dedicated datacenter or rotating residential + CapMonster for Turnstile + 1–2 req/sec with 15% jitter. Block rate under 15%. Residential proxies are overkill here unless the target uses Shopify Protect's advanced fingerprinting tier.

Cloudflare Standard (Business): Rotating residential + CapSolver or CapMonster for Turnstile + 0.5–1.5 req/sec with 40% jitter + curl-cffi or Camoufox for TLS fingerprinting. Block rate 15–25%. Add Camoufox if JS challenges are present.

Cloudflare Enterprise or Akamai Bot Manager: ISP or premium rotating residential + CapSolver (fastest Turnstile) + 0.3–1.0 req/sec + Camoufox with behavioral randomization. Block rate 25–40%. At this level, browser-level fingerprinting from the WAF bypass post is required alongside proxy and CAPTCHA infrastructure.

For e-commerce teams running competitor price monitoring at scale — tracking 50K+ SKUs across multiple retailers daily — the infrastructure cost of maintaining this stack (residential proxy spend, CAPTCHA solver credits, session management, retry logic) typically exceeds the cost of a managed scraping service within 2–3 months of operation.

When DIY Infrastructure Stops Making Sense

The test results above describe what's achievable with a well-configured DIY stack. They don't capture what it costs to keep it running.

Residential proxy pools degrade over time as IPs get flagged by target sites. CAPTCHA solver services change pricing and accuracy on Turnstile variants as Cloudflare updates its challenge implementation. Rate limiting thresholds on major retail targets tighten seasonally — Q4 is significantly more aggressive than Q1. A stack that achieves 85% pass rate in April may need reconfiguration in November.

This maintenance overhead is the actual cost of DIY anti-bot infrastructure. Engineering time spent on proxy rotation logic, CAPTCHA solver fallback chains, and rate limit response handlers is not spent on the data analysis that proxy access was meant to enable.

ScrapeWise handles the infrastructure layer — proxy management, CAPTCHA solving, rate adaptation — so that data extraction at scale is a configuration problem rather than an engineering project. For teams that want to maintain their own stack, the benchmarks above are the starting point. For teams that want the data without the infrastructure, start free on ScrapeWise.