Rate Limiting Explained

Every production API needs rate limiting. Without it, a single misbehaving client — or a deliberate attacker — can overwhelm your server with thousands of requests per second, crashing your application and affecting all users. Rate limiting controls how many requests each client can make within a time window, protecting your infrastructure while ensuring fair access for everyone.

This guide covers everything: what rate limiting is, the four major algorithms (and when to use each), HTTP 429 responses, rate limit headers, practical implementation in Node.js/Redis/NGINX, client-side handling, and real-world examples from GitHub, Stripe, and OpenAI.

What Is Rate Limiting?

Rate limiting is a technique that restricts the number of requests a client can make to an API within a specific time period. When a client exceeds the limit, the server rejects further requests with an HTTP 429 "Too Many Requests" response until the time window resets.

Real-world analogy: Think of a theme park with limited capacity. Only 100 visitors can enter per hour. The first 100 get in immediately. Visitor 101 is told to wait until the next hour. This prevents overcrowding (server overload) and ensures everyone inside has a good experience (acceptable response times).

Why Rate Limiting Matters

Rate Limiting Algorithms

There are four main algorithms for implementing rate limiting. Each has different trade-offs in accuracy, memory usage, burst handling, and implementation complexity.

1. Fixed Window Counter

The simplest algorithm. Divide time into fixed windows (e.g., 1-minute blocks). Count requests in the current window. If the count exceeds the limit, reject. When the window ends, reset to zero.

Problem: A client can send 100 requests at 11:59:59 and another 100 at 12:00:01 — 200 requests in 2 seconds, double the intended rate. This "boundary burst" is Fixed Window's weakness.

// Fixed Window in Redis
async function fixedWindowLimit(clientId, limit, windowSec) {
  const key = `ratelimit:${clientId}:${Math.floor(Date.now() / 1000 / windowSec)}`
  const count = await redis.incr(key)
  if (count === 1) await redis.expire(key, windowSec)
  return count <= limit
}

2. Sliding Window

Solves the boundary burst problem by considering a "sliding" time window that moves with the current time. Instead of fixed blocks, it looks at the last N seconds from right now. More accurate than Fixed Window but uses more memory (needs to track individual request timestamps).

The Sliding Window Counter variant approximates this by weighting the previous window's count. Example: if you're 30% through the current window, count = (current_count) + (previous_count × 0.7). This gives good accuracy with minimal memory.

3. Token Bucket

Imagine a bucket that holds tokens. Tokens are added at a fixed rate (e.g., 10 tokens/second). Each request consumes one token. If the bucket is empty, the request is rejected. If the bucket is full, extra tokens are discarded (max capacity = burst limit).

Token Bucket is the most popular algorithm because it naturally supports bursts. If a client hasn't made requests for a while, their bucket is full — they can send a burst of requests immediately. This matches real user behavior (idle periods followed by activity spikes).

// Token Bucket conceptual implementation
class TokenBucket {
  constructor(capacity, refillRate) {
    this.capacity = capacity       // Max tokens (burst limit)
    this.tokens = capacity         // Start full
    this.refillRate = refillRate   // Tokens per second
    this.lastRefill = Date.now()
  }

  consume() {
    this.refill()
    if (this.tokens >= 1) {
      this.tokens -= 1
      return true  // Request allowed
    }
    return false   // Rate limited (429)
  }

  refill() {
    const now = Date.now()
    const elapsed = (now - this.lastRefill) / 1000
    this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillRate)
    this.lastRefill = now
  }
}

4. Leaky Bucket

Imagine a bucket with a hole at the bottom. Requests flow in at any rate (filling the bucket), but they flow out (are processed) at a constant rate through the hole. If the bucket overflows (too many pending requests), new ones are rejected.

Leaky Bucket smooths traffic — regardless of how bursty the input is, output is always at a constant rate. This provides predictable server load but doesn't allow legitimate bursts. It's ideal for systems that need consistent throughput (video streaming, network queues).

Algorithm Comparison

Rate Limiting HTTP Headers

Well-designed APIs communicate rate limit status in response headers so clients can manage their request rate proactively:

// Standard rate limit response headers
HTTP/1.1 200 OK
RateLimit-Limit: 100              → Maximum requests per window
RateLimit-Remaining: 73           → Requests remaining in current window  
RateLimit-Reset: 1719216060       → Unix timestamp when window resets
X-RateLimit-Limit: 100           → (legacy non-standard version)
X-RateLimit-Remaining: 73        → (legacy non-standard version)

// When rate limited (429 response)
HTTP/1.1 429 Too Many Requests
Retry-After: 45                   → Seconds until client can retry
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 1719216060
Content-Type: application/json

{"error": "Rate limit exceeded", "retryAfter": 45}

Always include these headers — they let well-behaved clients back off before hitting the limit, reducing unnecessary 429 responses and improving overall API throughput.

Implementing Rate Limiting (Node.js + Redis)

// Express.js rate limiter using Redis (Token Bucket)
const Redis = require("ioredis")
const redis = new Redis()

async function rateLimit(req, res, next) {
  const clientId = req.ip // Or req.user.id for authenticated users
  const key = `ratelimit:${clientId}`
  const limit = 100          // Max tokens (burst capacity)
  const refillRate = 10      // Tokens per second
  const now = Date.now()

  // Atomic Redis operation
  const [tokens, lastRefill] = await redis.mget(
    `${key}:tokens`, `${key}:lastRefill`
  )

  let currentTokens = parseFloat(tokens) || limit
  const lastTime = parseInt(lastRefill) || now
  const elapsed = (now - lastTime) / 1000

  // Refill tokens
  currentTokens = Math.min(limit, currentTokens + elapsed * refillRate)

  if (currentTokens < 1) {
    const retryAfter = Math.ceil((1 - currentTokens) / refillRate)
    res.set("Retry-After", retryAfter)
    res.set("RateLimit-Limit", limit)
    res.set("RateLimit-Remaining", 0)
    return res.status(429).json({ error: "Too many requests", retryAfter })
  }

  // Consume token
  currentTokens -= 1
  await redis.mset(
    `${key}:tokens`, currentTokens,
    `${key}:lastRefill`, now
  )
  await redis.expire(`${key}:tokens`, 120)
  await redis.expire(`${key}:lastRefill`, 120)

  res.set("RateLimit-Limit", limit)
  res.set("RateLimit-Remaining", Math.floor(currentTokens))
  next()
}

// Apply to all routes
app.use(rateLimit)

// Or per-endpoint with different limits
app.post("/auth/login", rateLimit({ limit: 5, window: 60 }), loginHandler)

Client-Side: Handling 429 Responses

Good clients respect rate limits. Here is how to implement exponential backoff with jitter:

// Exponential backoff with jitter
async function fetchWithRetry(url, options, maxRetries = 3) {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    const response = await fetch(url, options)
    
    if (response.status !== 429) return response
    
    // Rate limited — wait and retry
    const retryAfter = response.headers.get("Retry-After")
    const waitTime = retryAfter 
      ? parseInt(retryAfter) * 1000
      : Math.min(1000 * Math.pow(2, attempt) + Math.random() * 1000, 30000)
    
    console.log(`Rate limited. Retrying in ${waitTime}ms (attempt ${attempt + 1})`)
    await new Promise(resolve => setTimeout(resolve, waitTime))
  }
  
  throw new Error("Max retries exceeded — still rate limited")
}

// Usage
const data = await fetchWithRetry("/api/users", { 
  headers: { "Authorization": "Bearer token..." }
})

Rate Limiting vs Throttling

Common Rate Limiting Mistakes

Best Practices

Real-World Rate Limiting Examples

Frequently Asked Questions

Related Articles & Tools