Rate Limiting a Public Extraction API in Next.js 15 App Router
Protect your zero-login developer playground from runaway scripts while giving legitimate SDK users transparent rate-limit headers.
Why this breaks in production
Offering a zero-login developer playground is one of the highest-converting PLG tactics for API startups — developers can test your endpoint in 5 seconds before ever signing up. Without rate limiting, however, a single misconfigured retry loop can hammer your serverless bill.
Returning standard `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, and `X-Extract-Time-Ms` headers makes your API feel production-grade and helps SDK clients implement polite exponential backoff.
Recommended Engineering Approach
Derive client key safely from `x-forwarded-for` / `x-real-ip`
Use the leading client IP address on Vercel/reverse proxies (and upgrade to API-key-scoped buckets for paid plans).
Return explicit rate-limit headers on every 200 and 429 response
Clients inspect `X-RateLimit-Remaining` proactively rather than waiting to hit HTTP 429.
Prune expired buckets to prevent unbounded memory growth
Clean out stale window entries opportunistically in warm serverless instances, and back with Upstash Redis when scaling multi-region.
Working Code Example
const res = await fetch("https://usecleansnap.com/api/v1/extract", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ url: "demo://docs" }),
});
console.log({
status: res.status,
limit: res.headers.get("x-ratelimit-limit"),
remaining: res.headers.get("x-ratelimit-remaining"),
extractMs: res.headers.get("x-extract-time-ms"),
});Test this workflow in the CleanSnap Playground
Pre-load the Sample Docs Page or paste your own URL to inspect clean Markdown, token savings, and auto-generated cURL / Node / Python / MCP code.
Common Edge Cases & Pitfalls
- In-memory Maps reset across cold starts and separate serverless regions — great for free playground throttling, but pair with Redis/Upstash for strict paid-tier metering.
- Always include `Cache-Control: no-store` on rate-limited API routes so CDNs do not accidentally cache a 429 response for other users.
- Expose helpful JSON error codes (`RATE_LIMIT_EXCEEDED`) alongside HTTP 429 so developer integrations can retry cleanly.
Frequently Asked Questions
What rate-limit headers does CleanSnap return?
Every `/api/v1/extract` call returns `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, and `X-Extract-Time-Ms`.
Can I configure the free rate limit via environment variables?
Yes — set `EXTRACT_RATE_LIMIT_PER_MIN` in `.env` when self-hosting CleanSnap.
How do Pro plans handle higher throughput?
Paid CleanSnap plans raise concurrency and requests-per-minute limits with dedicated API keys.