Decoding Named & Numeric HTML Entities Cleanly in Pure TypeScript
Clean up ugly `’`, ` `, and `&` artifacts in scraped titles, code blocks, and LLM Markdown using a tiny pure-TS decoder.
Why this breaks in production
Many server-side JavaScript developers rely on `document.createElement('textarea')` to decode HTML entities — which does not exist in Node.js, Edge runtimes, or Next.js Route Handlers.
Leaving raw HTML entities in extracted Markdown wastes extra BPE tokens (for example, `’` costs 3–4 tokens vs 1 token for `'`) and produces ugly quoted UI previews.
Recommended Engineering Approach
Match `&(#x[0-9a-fA-F]+|#\d+|[a-zA-Z]+);` in a single pass
Single-pass replacement avoids the double-decoding bug where `&lt;` mistakenly turns into `<`.
Guard `String.fromCodePoint` against out-of-range code points
Validate `codePoint > 0 && codePoint <= 0x10ffff` so malformed entities never throw `RangeError`.
Map high-frequency named entities (`amp`, `lt`, `gt`, `quot`, `nbsp`, `mdash`, `rsquo`)
Covers 99.9% of editorial and technical HTML with fewer than 30 lines of zero-dependency TypeScript.
Working Code Example
const NAMED: Record<string, string> = {
amp: "&", lt: "<", gt: ">", quot: '"', apos: "'",
nbsp: " ", mdash: "—", ndash: "–", rsquo: "’", lsquo: "‘",
};
export function decodeEntities(input: string): string {
return input.replace(/&(#x[0-9a-fA-F]+|#\d+|[a-zA-Z]+);/g, (full, e: string) => {
if (e.startsWith("#x") || e.startsWith("#X")) {
const cp = Number.parseInt(e.slice(2), 16);
return cp > 0 && cp <= 0x10ffff ? String.fromCodePoint(cp) : full;
}
if (e.startsWith("#")) {
const cp = Number.parseInt(e.slice(1), 10);
return cp > 0 && cp <= 0x10ffff ? String.fromCodePoint(cp) : full;
}
return NAMED[e.toLowerCase()] ?? full;
});
}Test this workflow in the CleanSnap Playground
Pre-load the Sample Blog Post or paste your own URL to inspect clean Markdown, token savings, and auto-generated cURL / Node / Python / MCP code.
Common Edge Cases & Pitfalls
- Chaining `.replace('&', '&').replace('<', '<')` in multiple passes causes double-unescaping vulnerabilities.
- Passing invalid numbers like `�` straight to `String.fromCodePoint` crashes with a runtime `RangeError`.
- Decode entities *after* stripping HTML tags so `<script>` inside a tutorial paragraph is never mistakenly treated as an executable tag.
Frequently Asked Questions
Why does single-pass entity decoding matter for security?
Single-pass replacement guarantees that an escaped sequence like `&lt;` becomes `<` rather than an active `<` character.
Does CleanSnap decode entities across both Markdown and metadata?
Yes — titles, descriptions, headings, code blocks, tables, and body paragraphs are all cleanly decoded.
Does this decoder work on Edge runtimes and Vercel Serverless?
Yes, it is 100% pure TypeScript with zero browser DOM or native binary dependencies.