Preserving Fenced Code Blocks & Language Hints in HTML-to-Markdown
Keep Python indentation, YAML spacing, and syntax language identifiers (`ts`, `bash`, `json`) 100% intact when converting HTML docs to Markdown.
Why this breaks in production
Most quick HTML-to-text scripts run `.replace(/\s+/g, ' ')` across the entire document. That destroys every `<pre><code>` snippet on the page — turning a readable 20-line Python script into a broken one-liner.
For coding agents and developer RAG systems, corrupted code blocks are fatal: LLMs copy broken syntax directly into user repositories.
Recommended Engineering Approach
Extract `<pre>` blocks before global whitespace cleanup
Swap each `<pre>...</pre>` match for an immutable placeholder token (`%%CODE_BLOCK_0%%`) stored in an array.
Detect language hints from `class="language-*"` or `lang-*`
Strip inner syntax-highlighting `<span>` tags (Prism, Shiki, Highlight.js), decode HTML entities (`<`, `>`), and wrap in fenced backticks.
Restore placeholders after paragraph normalization
Skip line trimming inside code fences so leading indentation in Python and YAML stays exact.
Working Code Example
const res = await fetch("https://usecleansnap.com/api/v1/extract", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
url: "demo://docs",
stripCookieBanners: true,
stripNav: true,
}),
});
const { markdown } = await res.json();
const codeBlocks = markdown.match(/```[a-z]*\n[\s\S]*?\n```/g) ?? [];
console.log(`Preserved ${codeBlocks.length} fenced code blocks cleanly!`);Test this workflow in the CleanSnap Playground
Pre-load the Sample Docs Page or paste your own URL to inspect clean Markdown, token savings, and auto-generated cURL / Node / Python / MCP code.
Common Edge Cases & Pitfalls
- Syntax highlighters like Shiki wrap every token in nested `<span>` tags — make sure inner HTML tags inside `<pre>` are stripped *without* inserting extra spaces.
- Double-decoding HTML entities inside code snippets can corrupt examples that intentionally show `&`.
- Leaving trailing blank lines inside fenced blocks creates noisy Markdown diffs.
Frequently Asked Questions
Does CleanSnap work with Prism.js, Highlight.js, and Shiki code blocks?
Yes. CleanSnap strips inner highlight `<span>` tags cleanly, decodes entities, preserves indentation, and detects `language-*` / `lang-*` fence hints.
What about inline `<code>` elements inside paragraphs?
Inline `<code>` tags are converted to single-backtick Markdown (`` `stream: true` ``) separately from multi-line `<pre>` fences.
Can I preview code block preservation right now?
Yes — run the 'Sample Docs Page' demo preset in the Playground on the homepage.