CleanSnapv1 API
All 25 Developer GuidesMetadata & Parsing
Intermediate·5 min read·Topic: preserve code blocks html to markdown

Preserving Fenced Code Blocks & Language Hints in HTML-to-Markdown

Keep Python indentation, YAML spacing, and syntax language identifiers (`ts`, `bash`, `json`) 100% intact when converting HTML docs to Markdown.

Why this breaks in production

Most quick HTML-to-text scripts run `.replace(/\s+/g, ' ')` across the entire document. That destroys every `<pre><code>` snippet on the page — turning a readable 20-line Python script into a broken one-liner.

For coding agents and developer RAG systems, corrupted code blocks are fatal: LLMs copy broken syntax directly into user repositories.

Recommended Engineering Approach

1

Extract `<pre>` blocks before global whitespace cleanup

Swap each `<pre>...</pre>` match for an immutable placeholder token (`%%CODE_BLOCK_0%%`) stored in an array.

2

Detect language hints from `class="language-*"` or `lang-*`

Strip inner syntax-highlighting `<span>` tags (Prism, Shiki, Highlight.js), decode HTML entities (`&lt;`, `&gt;`), and wrap in fenced backticks.

3

Restore placeholders after paragraph normalization

Skip line trimming inside code fences so leading indentation in Python and YAML stays exact.

Working Code Example

verify-code-fences.tsts
const res = await fetch("https://usecleansnap.com/api/v1/extract", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    url: "demo://docs",
    stripCookieBanners: true,
    stripNav: true,
  }),
});

const { markdown } = await res.json();
const codeBlocks = markdown.match(/```[a-z]*\n[\s\S]*?\n```/g) ?? [];
console.log(`Preserved ${codeBlocks.length} fenced code blocks cleanly!`);
TRY IT LIVE — ZERO LOGIN

Test this workflow in the CleanSnap Playground

Pre-load the Sample Docs Page or paste your own URL to inspect clean Markdown, token savings, and auto-generated cURL / Node / Python / MCP code.

Run Sample Docs Page

Common Edge Cases & Pitfalls

  • Syntax highlighters like Shiki wrap every token in nested `<span>` tags — make sure inner HTML tags inside `<pre>` are stripped *without* inserting extra spaces.
  • Double-decoding HTML entities inside code snippets can corrupt examples that intentionally show `&amp;`.
  • Leaving trailing blank lines inside fenced blocks creates noisy Markdown diffs.

Frequently Asked Questions

Does CleanSnap work with Prism.js, Highlight.js, and Shiki code blocks?

Yes. CleanSnap strips inner highlight `<span>` tags cleanly, decodes entities, preserves indentation, and detects `language-*` / `lang-*` fence hints.

What about inline `<code>` elements inside paragraphs?

Inline `<code>` tags are converted to single-backtick Markdown (`` `stream: true` ``) separately from multi-line `<pre>` fences.

Can I preview code block preservation right now?

Yes — run the 'Sample Docs Page' demo preset in the Playground on the homepage.

Related Developer Guides