CleanSnapv1 API
Beginner·5 min read·Topic: webpage to markdown for llm

Converting Any Webpage to Clean LLM-Ready Markdown

Transform noisy HTML documents into semantic, token-efficient Markdown that preserves headings, code blocks, lists, and links for GPT, Claude, and Gemini.

Why this breaks in production

Modern LLMs are heavily pre-trained on GitHub READMEs, technical documentation, and Markdown corpora. Feeding raw HTML wastes 70%–90% of your context window on Tailwind utility classes, `<svg>` icon paths, and inline hydration scripts.

Conversely, stripping HTML down to flat plain text (`innerText`) destroys hierarchy: headings merge into paragraphs, Markdown tables flatten into unreadable strings, and code indentation vanishes. Clean semantic Markdown hits the exact sweet spot.

Recommended Engineering Approach

1

Isolate main content before conversion

Remove `<script>`, `<style>`, `<svg>`, `<nav>`, and `<footer>`, then prefer `<article>` or `<main>` when sufficient body copy is present.

2

Protect `<pre><code>` blocks early

Replace fenced code blocks with temporary placeholders prior to whitespace normalization so indentation and language hints survive intact.

3

Emit GFM tables, headings, and resolved absolute links

Resolve relative `/docs/auth` links against the page's canonical base URL so your AI agent can safely follow citations.

Working Code Example

fetch-llm-markdown.tsts
export async function fetchPageForLLM(url: string) {
  const response = await fetch("https://usecleansnap.com/api/v1/extract", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      url,
      stripCookieBanners: true,
      stripNav: true,
      includeMetadata: true,
      maxChars: 16000,
    }),
  });

  if (!response.ok) {
    throw new Error(`Extraction failed: ${response.status}`);
  }

  const data = await response.json();
  return {
    promptContext: `# ${data.metadata.title}\nSource: ${data.finalUrl}\n\n${data.markdown}`,
    tokens: data.stats.estimatedTokens,
    savedPct: data.stats.tokenSavingsPercent,
  };
}
TRY IT LIVE — ZERO LOGIN

Test this workflow in the CleanSnap Playground

Pre-load the Sample Blog Post or paste your own URL to inspect clean Markdown, token savings, and auto-generated cURL / Node / Python / MCP code.

Run Sample Blog Post

Common Edge Cases & Pitfalls

  • Forgetting to resolve relative links (`/pricing`) leaves AI agents hallucinating domains when asked to cite source URLs.
  • Unbounded pages (e.g., giant changelogs) can overflow an agent's context budget unless you enforce `maxChars`.
  • Leaving HTML entities like `&amp;` or `&#8217;` un-decoded degrades tokenization efficiency and produces messy quoted answers.

Frequently Asked Questions

Why is Markdown better than plain text for LLMs?

Markdown preserves heading levels (`#`, `##`), lists, blockquotes, tables, and code fences with minimal token overhead, helping LLMs understand document structure and provenance.

How much token reduction can I expect from HTML to Clean Markdown?

Across typical SaaS docs and blog posts, converting raw HTML to CleanSnap Markdown reduces estimated tokens by 75% to 92%.

Can I cap the output length for smaller context windows?

Yes — pass `maxChars` (for example `12000` for ~3,000 tokens) and CleanSnap cleanly truncates the output and sets `stats.truncated = true`.

Related Developer Guides