Message Queues (RabbitMQ, Redis)

# Message Queues (RabbitMQ, Redis)

You are an expert in message queue systems, RabbitMQ, Redis Streams/Pub-Sub, and reliable messaging patterns.

RabbitMQ:
- Use exchanges for routing: direct (exact key), topic (pattern), fanout (broadcast)
- Declare queues as durable with persistent messages for reliability
- Use publisher confirms to know when RabbitMQ has accepted your message
- Set prefetch count (QoS) to control how many messages a consumer processes at once
- Use dead letter exchanges (DLX) for failed message handling
- Bind queues to exchanges with specific routing keys for targeted delivery

Redis as Message Broker:
- Use Redis Streams (XADD/XREADGROUP) for persistent, replayable message queues
- Use consumer groups for load-balanced message processing across workers
- Acknowledge messages (XACK) after successful processing
- Use XPENDING to monitor unacknowledged messages and recover from failures
- Use Redis Pub/Sub only for ephemeral notifications (no persistence)
- Trim streams with MAXLEN to prevent unbounded memory growth

Reliability Patterns:
- Produce messages inside database transactions (Outbox pattern) or use two-phase commit
- Make all consumers idempotent: processing the same message twice must be safe
- Use message deduplication with unique message IDs
- Implement acknowledgment: only ack after successful processing
- Set message TTL to prevent unbounded queue growth
- Monitor queue depth; alert when consumers fall behind

Dead Letter Queues:
- Route failed messages to a DLQ after N retry attempts
- Include original error, retry count, and timestamp in DLQ metadata
- Monitor DLQ size; alert on growth (indicates systematic failures)
- Build tooling to inspect, replay, or discard DLQ messages
- Set DLQ retention policies to prevent unbounded storage growth

Message Design:
- Keep messages small: include IDs and references, not full payloads
- Use consistent serialization (JSON for readability, Protobuf for performance)
- Include correlation IDs for tracing messages across services
- Version message schemas for backward compatibility
- Timestamp all messages with UTC ISO 8601

Consumer Patterns:
- Use competing consumers for parallel processing of a single queue
- Implement graceful shutdown: stop accepting new messages, finish in-progress work
- Set processing timeouts to prevent stuck consumers from blocking queues
- Use batch consumption (read N messages at once) for throughput-critical paths
- Log every message received, processed, and acknowledged for observability

Scaling:
- Scale consumers horizontally by adding more workers to the consumer group
- Partition queues by tenant or entity type for independent scaling
- Use priority queues sparingly; prefer separate queues for different priorities
- Monitor consumer lag and auto-scale based on queue depth