# Producer-Consumer Pattern

## Problem

One or more components generate data (producers) while other components process that data (consumers). Without coordination, producers may overwhelm consumers or consumers may busy-wait when no data is available.

## Solution

Introduce a shared bounded queue between producers and consumers. Producers enqueue items; consumers dequeue them. The queue handles synchronization: producers block when the queue is full (backpressure), and consumers block when the queue is empty.

## When to Use

- Decoupling data generation from processing stages
- Implementing pipeline architectures
- Buffering bursts of incoming work
- Rate-limiting producers to match consumer throughput

## When to Avoid

- Single-threaded applications with no concurrency needs
- When data must be processed in strict order without buffering
- Extremely low-latency paths where queue overhead is unacceptable

## Pseudocode

```
queue = BoundedQueue(capacity=10)

function producer(id):
    for i in 1..5:
        item = "item-" + id + "-" + i
        queue.put(item)          // blocks if full
        print("Produced: " + item)

function consumer(id):
    while true:
        item = queue.get()       // blocks if empty
        if item == SENTINEL:
            break
        print("Consumer " + id + " processed: " + item)

// Launch producers and consumers, then send sentinel values to stop
```

## Python

```python
import threading
import queue
import time

def producer(q: queue.Queue, producer_id: int) -> None:
    for i in range(5):
        item = f"item-{producer_id}-{i}"
        q.put(item)
        print(f"[Producer {producer_id}] Produced: {item}")
        time.sleep(0.05)

def consumer(q: queue.Queue, consumer_id: int) -> None:
    while True:
        item = q.get()
        if item is None:  # sentinel
            q.task_done()
            break
        print(f"[Consumer {consumer_id}] Processed: {item}")
        time.sleep(0.1)
        q.task_done()

def main() -> None:
    q: queue.Queue = queue.Queue(maxsize=5)

    producers = [threading.Thread(target=producer, args=(q, i)) for i in range(2)]
    consumers = [threading.Thread(target=consumer, args=(q, i)) for i in range(3)]

    for c in consumers:
        c.start()
    for p in producers:
        p.start()

    for p in producers:
        p.join()

    # Send sentinel for each consumer
    for _ in consumers:
        q.put(None)

    for c in consumers:
        c.join()

    print("All done.")

if __name__ == "__main__":
    main()
```

## Go

```go
package main

import (
	"fmt"
	"sync"
	"time"
)

func producer(id int, ch chan<- string, wg *sync.WaitGroup) {
	defer wg.Done()
	for i := 0; i < 5; i++ {
		item := fmt.Sprintf("item-%d-%d", id, i)
		ch <- item
		fmt.Printf("[Producer %d] Produced: %s\n", id, item)
		time.Sleep(50 * time.Millisecond)
	}
}

func consumer(id int, ch <-chan string, wg *sync.WaitGroup) {
	defer wg.Done()
	for item := range ch {
		fmt.Printf("[Consumer %d] Processed: %s\n", id, item)
		time.Sleep(100 * time.Millisecond)
	}
}

func main() {
	ch := make(chan string, 5)

	var prodWg, consWg sync.WaitGroup

	for i := 0; i < 2; i++ {
		prodWg.Add(1)
		go producer(i, ch, &prodWg)
	}

	for i := 0; i < 3; i++ {
		consWg.Add(1)
		go consumer(i, ch, &consWg)
	}

	prodWg.Wait()
	close(ch) // signals consumers to finish
	consWg.Wait()

	fmt.Println("All done.")
}
```

## JavaScript

```javascript
class AsyncQueue {
  constructor(capacity) {
    this.capacity = capacity;
    this.items = [];
    this.waitingConsumers = [];
    this.waitingProducers = [];
  }

  async put(item) {
    while (this.items.length >= this.capacity) {
      await new Promise((resolve) => this.waitingProducers.push(resolve));
    }
    this.items.push(item);
    if (this.waitingConsumers.length > 0) {
      this.waitingConsumers.shift()();
    }
  }

  async get() {
    while (this.items.length === 0) {
      await new Promise((resolve) => this.waitingConsumers.push(resolve));
    }
    const item = this.items.shift();
    if (this.waitingProducers.length > 0) {
      this.waitingProducers.shift()();
    }
    return item;
  }
}

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function producer(queue, id) {
  for (let i = 0; i < 5; i++) {
    const item = `item-${id}-${i}`;
    await queue.put(item);
    console.log(`[Producer ${id}] Produced: ${item}`);
    await sleep(50);
  }
}

async function consumer(queue, id, count) {
  for (let i = 0; i < count; i++) {
    const item = await queue.get();
    console.log(`[Consumer ${id}] Processed: ${item}`);
    await sleep(100);
  }
}

async function main() {
  const queue = new AsyncQueue(5);
  const totalItems = 10; // 2 producers * 5 items
  const numConsumers = 3;
  const perConsumer = Math.ceil(totalItems / numConsumers);

  const producers = [producer(queue, 0), producer(queue, 1)];
  const consumers = [];
  let remaining = totalItems;
  for (let i = 0; i < numConsumers; i++) {
    const count = Math.min(perConsumer, remaining);
    remaining -= count;
    consumers.push(consumer(queue, i, count));
  }

  await Promise.all([...producers, ...consumers]);
  console.log("All done.");
}

main();
```

## Related Patterns

- **Message Queue** — a message queue is the durable, distributed version of the producer-consumer bounded buffer.
- **Thread Pool** — thread pool manages consumer workers; producer-consumer defines how work flows into those workers.
- **Monitor** — the bounded buffer in producer-consumer is typically implemented using a monitor (mutex + condition variables).
- **Bulkhead** — bulkhead limits concurrent consumers per dependency; producer-consumer controls flow between stages.
