# Rate Limiter Pattern

## Problem

Without throttling, a service can be overwhelmed by too many requests — whether from a single greedy client, a burst of traffic, or a denial-of-service attack. Uncontrolled throughput leads to degraded performance for all users and potential service crashes.

## Solution

Implement a **token bucket** algorithm: a bucket holds tokens up to a maximum capacity. Each request consumes one token. Tokens are replenished at a fixed rate. When the bucket is empty, requests are rejected (or queued). This allows short bursts while enforcing a sustained average rate.

Variants:
- **Token Bucket** — Allows bursts up to bucket capacity.
- **Leaky Bucket** — Smooths output to a constant rate.
- **Sliding Window** — Counts requests in a rolling time window.

## When to Use

- Public APIs that need to enforce usage quotas
- Protecting services from traffic spikes or abuse
- Multi-tenant systems where fairness between tenants matters
- Upstream dependencies with known throughput limits

## When to Avoid

- Internal service-to-service calls with trusted, controlled traffic
- Batch processing pipelines where throughput is the goal
- When backpressure mechanisms already exist (e.g., message queue)
- Extremely latency-sensitive paths where the check overhead matters

## Pseudocode

```
class TokenBucket:
    capacity = N
    tokens = N
    refill_rate = R tokens/second
    last_refill = now()

    function allow_request():
        refill()
        if tokens >= 1:
            tokens -= 1
            return true
        return false

    function refill():
        elapsed = now() - last_refill
        new_tokens = elapsed * refill_rate
        tokens = min(capacity, tokens + new_tokens)
        last_refill = now()
```

## Python Implementation

```python
import time
import threading


class TokenBucket:
    def __init__(self, capacity: int, refill_rate: float):
        self.capacity = capacity
        self.refill_rate = refill_rate  # tokens per second
        self.tokens = float(capacity)
        self.last_refill = time.monotonic()
        self._lock = threading.Lock()

    def _refill(self):
        now = time.monotonic()
        elapsed = now - self.last_refill
        new_tokens = elapsed * self.refill_rate
        self.tokens = min(self.capacity, self.tokens + new_tokens)
        self.last_refill = now

    def allow(self) -> bool:
        with self._lock:
            self._refill()
            if self.tokens >= 1.0:
                self.tokens -= 1.0
                return True
            return False

    @property
    def available(self) -> float:
        with self._lock:
            self._refill()
            return self.tokens


class RateLimiter:
    def __init__(self, capacity: int, refill_rate: float):
        self.bucket = TokenBucket(capacity, refill_rate)

    def execute(self, operation):
        if not self.bucket.allow():
            raise RateLimitExceeded(
                f"Rate limit exceeded. Available tokens: {self.bucket.available:.1f}"
            )
        return operation()


class RateLimitExceeded(Exception):
    pass


# --- Demo ---
limiter = RateLimiter(capacity=5, refill_rate=2.0)  # 5 burst, 2/sec sustained

print("=== Burst of 8 requests (capacity=5) ===")
for i in range(1, 9):
    try:
        result = limiter.execute(lambda i=i: f"Request {i} processed")
        print(f"  Request {i}: ALLOWED — {result}")
    except RateLimitExceeded:
        print(f"  Request {i}: DENIED — rate limit exceeded")

print(f"\n  Waiting 2 seconds for tokens to refill...")
time.sleep(2)

print(f"\n=== After refill (should have ~4 tokens) ===")
for i in range(9, 15):
    try:
        result = limiter.execute(lambda i=i: f"Request {i} processed")
        print(f"  Request {i}: ALLOWED — {result}")
    except RateLimitExceeded:
        print(f"  Request {i}: DENIED — rate limit exceeded")
```

**Output:**
```
=== Burst of 8 requests (capacity=5) ===
  Request 1: ALLOWED — Request 1 processed
  Request 2: ALLOWED — Request 2 processed
  Request 3: ALLOWED — Request 3 processed
  Request 4: ALLOWED — Request 4 processed
  Request 5: ALLOWED — Request 5 processed
  Request 6: DENIED — rate limit exceeded
  Request 7: DENIED — rate limit exceeded
  Request 8: DENIED — rate limit exceeded

  Waiting 2 seconds for tokens to refill...

=== After refill (should have ~4 tokens) ===
  Request 9: ALLOWED — Request 9 processed
  Request 10: ALLOWED — Request 10 processed
  Request 11: ALLOWED — Request 11 processed
  Request 12: ALLOWED — Request 12 processed
  Request 13: DENIED — rate limit exceeded
  Request 14: DENIED — rate limit exceeded
```

## Go Implementation

```go
package main

import (
	"fmt"
	"math"
	"sync"
	"time"
)

type TokenBucket struct {
	mu         sync.Mutex
	capacity   float64
	tokens     float64
	refillRate float64 // tokens per second
	lastRefill time.Time
}

func NewTokenBucket(capacity int, refillRate float64) *TokenBucket {
	return &TokenBucket{
		capacity:   float64(capacity),
		tokens:     float64(capacity),
		refillRate: refillRate,
		lastRefill: time.Now(),
	}
}

func (tb *TokenBucket) refill() {
	now := time.Now()
	elapsed := now.Sub(tb.lastRefill).Seconds()
	tb.tokens = math.Min(tb.capacity, tb.tokens+elapsed*tb.refillRate)
	tb.lastRefill = now
}

func (tb *TokenBucket) Allow() bool {
	tb.mu.Lock()
	defer tb.mu.Unlock()
	tb.refill()
	if tb.tokens >= 1.0 {
		tb.tokens -= 1.0
		return true
	}
	return false
}

func main() {
	bucket := NewTokenBucket(5, 2.0)

	fmt.Println("=== Burst of 8 requests (capacity=5) ===")
	for i := 1; i <= 8; i++ {
		if bucket.Allow() {
			fmt.Printf("  Request %d: ALLOWED\n", i)
		} else {
			fmt.Printf("  Request %d: DENIED\n", i)
		}
	}

	fmt.Println("\n  Waiting 2 seconds for refill...")
	time.Sleep(2 * time.Second)

	fmt.Println("\n=== After refill ===")
	for i := 9; i <= 14; i++ {
		if bucket.Allow() {
			fmt.Printf("  Request %d: ALLOWED\n", i)
		} else {
			fmt.Printf("  Request %d: DENIED\n", i)
		}
	}
}
```

## JavaScript Implementation

```javascript
class TokenBucket {
  constructor(capacity, refillRate) {
    this.capacity = capacity;
    this.tokens = capacity;
    this.refillRate = refillRate; // tokens per second
    this.lastRefill = Date.now();
  }

  _refill() {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillRate);
    this.lastRefill = now;
  }

  allow() {
    this._refill();
    if (this.tokens >= 1) {
      this.tokens -= 1;
      return true;
    }
    return false;
  }
}

// --- Demo ---
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

(async () => {
  const bucket = new TokenBucket(5, 2.0);

  console.log("=== Burst of 8 requests (capacity=5) ===");
  for (let i = 1; i <= 8; i++) {
    if (bucket.allow()) {
      console.log(`  Request ${i}: ALLOWED`);
    } else {
      console.log(`  Request ${i}: DENIED`);
    }
  }

  console.log("\n  Waiting 2 seconds for refill...");
  await sleep(2000);

  console.log("\n=== After refill ===");
  for (let i = 9; i <= 14; i++) {
    if (bucket.allow()) {
      console.log(`  Request ${i}: ALLOWED`);
    } else {
      console.log(`  Request ${i}: DENIED`);
    }
  }
})();
```

## Related Patterns

- **Bulkhead** — rate limiter controls the request rate over time; bulkhead limits concurrency at any instant.
- **Monitor** — use a monitor to observe rate-limiter metrics (rejection rate, token level) and alert on anomalies.
- **Proxy** — a rate-limiting proxy intercepts all requests before they reach the backend service.
- **Token Bucket** — the token bucket algorithm is the most common implementation of rate limiting.
