CleanSnapv1 API
All 25 Developer GuidesSecurity & Production
Intermediate·6 min read·Topic: rate limit public extraction api nextjs

Rate Limiting a Public Extraction API in Next.js 15 App Router

Protect your zero-login developer playground from runaway scripts while giving legitimate SDK users transparent rate-limit headers.

Why this breaks in production

Offering a zero-login developer playground is one of the highest-converting PLG tactics for API startups — developers can test your endpoint in 5 seconds before ever signing up. Without rate limiting, however, a single misconfigured retry loop can hammer your serverless bill.

Returning standard `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, and `X-Extract-Time-Ms` headers makes your API feel production-grade and helps SDK clients implement polite exponential backoff.

Recommended Engineering Approach

1

Derive client key safely from `x-forwarded-for` / `x-real-ip`

Use the leading client IP address on Vercel/reverse proxies (and upgrade to API-key-scoped buckets for paid plans).

2

Return explicit rate-limit headers on every 200 and 429 response

Clients inspect `X-RateLimit-Remaining` proactively rather than waiting to hit HTTP 429.

3

Prune expired buckets to prevent unbounded memory growth

Clean out stale window entries opportunistically in warm serverless instances, and back with Upstash Redis when scaling multi-region.

Working Code Example

inspect-rate-limit-headers.tsts
const res = await fetch("https://usecleansnap.com/api/v1/extract", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ url: "demo://docs" }),
});

console.log({
  status: res.status,
  limit: res.headers.get("x-ratelimit-limit"),
  remaining: res.headers.get("x-ratelimit-remaining"),
  extractMs: res.headers.get("x-extract-time-ms"),
});
TRY IT LIVE — ZERO LOGIN

Test this workflow in the CleanSnap Playground

Pre-load the Sample Docs Page or paste your own URL to inspect clean Markdown, token savings, and auto-generated cURL / Node / Python / MCP code.

Run Sample Docs Page

Common Edge Cases & Pitfalls

  • In-memory Maps reset across cold starts and separate serverless regions — great for free playground throttling, but pair with Redis/Upstash for strict paid-tier metering.
  • Always include `Cache-Control: no-store` on rate-limited API routes so CDNs do not accidentally cache a 429 response for other users.
  • Expose helpful JSON error codes (`RATE_LIMIT_EXCEEDED`) alongside HTTP 429 so developer integrations can retry cleanly.

Frequently Asked Questions

What rate-limit headers does CleanSnap return?

Every `/api/v1/extract` call returns `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, and `X-Extract-Time-Ms`.

Can I configure the free rate limit via environment variables?

Yes — set `EXTRACT_RATE_LIMIT_PER_MIN` in `.env` when self-hosting CleanSnap.

How do Pro plans handle higher throughput?

Paid CleanSnap plans raise concurrency and requests-per-minute limits with dedicated API keys.

Related Developer Guides