Converting Any Webpage to Clean LLM-Ready Markdown
Transform noisy HTML documents into semantic, token-efficient Markdown that preserves headings, code blocks, lists, and links for GPT, Claude, and Gemini.
Why this breaks in production
Modern LLMs are heavily pre-trained on GitHub READMEs, technical documentation, and Markdown corpora. Feeding raw HTML wastes 70%–90% of your context window on Tailwind utility classes, `<svg>` icon paths, and inline hydration scripts.
Conversely, stripping HTML down to flat plain text (`innerText`) destroys hierarchy: headings merge into paragraphs, Markdown tables flatten into unreadable strings, and code indentation vanishes. Clean semantic Markdown hits the exact sweet spot.
Recommended Engineering Approach
Isolate main content before conversion
Remove `<script>`, `<style>`, `<svg>`, `<nav>`, and `<footer>`, then prefer `<article>` or `<main>` when sufficient body copy is present.
Protect `<pre><code>` blocks early
Replace fenced code blocks with temporary placeholders prior to whitespace normalization so indentation and language hints survive intact.
Emit GFM tables, headings, and resolved absolute links
Resolve relative `/docs/auth` links against the page's canonical base URL so your AI agent can safely follow citations.
Working Code Example
export async function fetchPageForLLM(url: string) {
const response = await fetch("https://usecleansnap.com/api/v1/extract", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
url,
stripCookieBanners: true,
stripNav: true,
includeMetadata: true,
maxChars: 16000,
}),
});
if (!response.ok) {
throw new Error(`Extraction failed: ${response.status}`);
}
const data = await response.json();
return {
promptContext: `# ${data.metadata.title}\nSource: ${data.finalUrl}\n\n${data.markdown}`,
tokens: data.stats.estimatedTokens,
savedPct: data.stats.tokenSavingsPercent,
};
}Test this workflow in the CleanSnap Playground
Pre-load the Sample Blog Post or paste your own URL to inspect clean Markdown, token savings, and auto-generated cURL / Node / Python / MCP code.
Common Edge Cases & Pitfalls
- Forgetting to resolve relative links (`/pricing`) leaves AI agents hallucinating domains when asked to cite source URLs.
- Unbounded pages (e.g., giant changelogs) can overflow an agent's context budget unless you enforce `maxChars`.
- Leaving HTML entities like `&` or `’` un-decoded degrades tokenization efficiency and produces messy quoted answers.
Frequently Asked Questions
Why is Markdown better than plain text for LLMs?
Markdown preserves heading levels (`#`, `##`), lists, blockquotes, tables, and code fences with minimal token overhead, helping LLMs understand document structure and provenance.
How much token reduction can I expect from HTML to Clean Markdown?
Across typical SaaS docs and blog posts, converting raw HTML to CleanSnap Markdown reduces estimated tokens by 75% to 92%.
Can I cap the output length for smaller context windows?
Yes — pass `maxChars` (for example `12000` for ~3,000 tokens) and CleanSnap cleanly truncates the output and sets `stats.truncated = true`.