CleanSnapv1 API
All 25 Developer GuidesCookie & Clutter Removal
Intermediate·6 min read·Topic: hide cookie banner puppeteer

How to Hide & Strip Cookie Banners in Puppeteer (or Skip Headless Chrome Entirely)

Remove OneTrust, Cookiebot, TrustArc, and custom GDPR overlays cleanly before extracting text or feeding pages to an LLM.

Why this breaks in production

Every EU-compliant site ships a consent management platform (CMP) like OneTrust, Cookiebot, Didomi, or Quantcast. In headless Puppeteer runs, these modals inject 200–600 words of legal boilerplate directly into document.body.innerText and often trap scroll focus.

Clicking 'Accept All' is brittle because button labels change by locale ('Accepter tout', 'Alle akzeptieren') and shadow DOM containers resist basic querySelector clicks. Removing the consent nodes at the DOM/HTML layer — or offloading extraction to a specialized API — is dramatically faster and deterministic.

Recommended Engineering Approach

1

Match known CMP containers by ID, class, and ARIA role

Target common identifiers (`onetrust-banner-sdk`, `CybotCookiebotDialog`, `truste-consent-track`, `[role='dialog'][aria-label*='cookie' i]`) before reading page text.

2

Block CMP vendor scripts at the network layer if using Puppeteer

Abort requests to `cdn.cookielaw.org`, `consent.cookiebot.com`, and `consent.trustarc.com` via `page.setRequestInterception(true)` so banners never hydrate.

3

Use CleanSnap API for SSR pages without launching Chromium

For docs, blogs, and SaaS pages that already render HTML server-side, call `/api/v1/extract` with `stripCookieBanners: true` to excise balanced consent containers in ~300ms.

Working Code Example

puppeteer-vs-cleansnap-cookie-strip.tsts
// Option A: Strip cookie banners in Puppeteer before reading DOM
await page.evaluate(() => {
  const selectors = [
    "#onetrust-banner-sdk",
    "#CybotCookiebotDialog",
    "[id*='cookie' i][role='dialog']",
    "[class*='cookie-banner' i]",
    "[class*='gdpr-modal' i]",
  ];
  document.querySelectorAll(selectors.join(",")).forEach((el) => el.remove());
});

// Option B: Zero-browser CleanSnap API call (10x faster on SSR/SSG pages)
const res = await fetch("https://usecleansnap.com/api/v1/extract", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    url: "https://example.com/docs",
    stripCookieBanners: true,
    stripNav: true,
  }),
});
const { markdown, stats } = await res.json();
console.log("Stripped:", stats.strippedElements);
TRY IT LIVE — ZERO LOGIN

Test this workflow in the CleanSnap Playground

Pre-load the Sample Docs Page or paste your own URL to inspect clean Markdown, token savings, and auto-generated cURL / Node / Python / MCP code.

Run Sample Docs Page

Common Edge Cases & Pitfalls

  • Naive regex removal of `<div class='cookie...'>` breaks when consent banners contain nested `<div>` elements — always use balanced tag matching or DOM removal.
  • Some CMPs inject `overflow: hidden` onto `<body>`; if taking screenshots in Puppeteer, reset `document.body.style.overflow = 'auto'`.
  • Do not match the word `cookie` across the entire body text or you will accidentally delete baking recipes and HTTP cookie tutorials — inspect element attributes only.

Frequently Asked Questions

Why does clicking 'Accept All' fail in Puppeteer across European sites?

Consent buttons are frequently localized into 20+ languages, embedded inside cross-origin iframes (Sourcepoint, Didomi), or delayed by 1–2 seconds after `domcontentloaded`.

Does CleanSnap remove cookie banners without headless Chrome?

Yes. For server-rendered consent banners embedded in HTML, CleanSnap removes balanced consent blocks automatically when `stripCookieBanners: true` is passed.

How many LLM tokens does a typical GDPR modal waste?

Between 180 and 550 tokens per URL — which compounds rapidly when an AI agent inspects 10–20 URLs per research task.

Related Developer Guides