Stripping OneTrust, Cookiebot & TrustArc GDPR Overlays from Raw HTML
Why simple regex fails on nested consent `<div>` trees and how balanced-depth HTML excision cleanly removes CMP boilerplate.
Why this breaks in production
Enterprise sites embed massive OneTrust (`#onetrust-banner-sdk`), Cookiebot (`#CybotCookiebotDialog`), or TrustArc (`#truste-consent-track`) trees containing 15+ nested `<div>` wrappers for vendor toggles and privacy links.
A naive regex like `/<div[^>]*onetrust[^>]*>[\s\S]*?<\/div>/i` stops at the very first inner `</div>`, leaving orphaned consent text and broken HTML trailing across your output.
Recommended Engineering Approach
Match CMP attributes on container tags (`div`, `section`, `aside`, `dialog`)
Inspect `id`, `class`, `role`, `aria-label`, and `data-testid` for CMP signatures (`onetrust`, `cookiebot`, `trustarc`, `didomi`, `quantcast`, `gdpr`, `consent`).
Walk opening and closing tags with a nesting depth counter
Increment depth on each nested `<div...>` and decrement on `</div>` until depth hits zero, ensuring the entire consent subtree is excised cleanly.
Guard against malformed unclosed tags
If a malformed page never closes the tag, fall back gracefully rather than deleting the rest of the webpage.
Working Code Example
// Call CleanSnap with stripCookieBanners toggled on vs off to compare
async function compareCookieStrip(url: string) {
const [clean, raw] = await Promise.all([
fetch("https://usecleansnap.com/api/v1/extract", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ url, stripCookieBanners: true, stripNav: true }),
}).then((r) => r.json()),
fetch("https://usecleansnap.com/api/v1/extract", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ url, stripCookieBanners: false, stripNav: true }),
}).then((r) => r.json()),
]);
console.log("Tokens saved by cookie stripping:", raw.stats.estimatedTokens - clean.stats.estimatedTokens);
}Test this workflow in the CleanSnap Playground
Pre-load the Sample Docs Page or paste your own URL to inspect clean Markdown, token savings, and auto-generated cURL / Node / Python / MCP code.
Common Edge Cases & Pitfalls
- Non-greedy regexes fail immediately on nested `<div>` trees inside OneTrust and Cookiebot overlays.
- Self-closing tags (`<div />` in XHTML) must not increment nesting depth.
- Avoid matching overly broad class words like `modal` alone, which could remove legitimate documentation callouts.
Frequently Asked Questions
How can I verify that CleanSnap actually removed the cookie banner?
Inspect `stats.strippedElements` in the JSON response — it reports exact counts such as `cookie/consent banners (1)`.
Can I disable cookie stripping for compliance auditing?
Yes — set `stripCookieBanners: false` if you intentionally want to capture the site's privacy notice text.
Does CleanSnap handle Cookiebot and TrustArc as well as OneTrust?
Yes, all major CMP container naming conventions are matched automatically.