Redis Explained for Developers

Data structures, use cases, persistence, clustering & best practices

API DevelopmentJuly 4, 202620 min readBy Keyur Patel

You need sub-millisecond reads. You need session storage that survives server restarts. You need a pub/sub system that fans out messages to thousands of subscribers. Redis does all of it — and it's probably already running somewhere in your infrastructure.

Most developers first encounter Redis as "that cache layer" sitting between their API and the database. But Redis is far more than a cache. It's a data structure server that handles everything from rate limiting to real-time leaderboards to event streaming. Once you understand what Redis actually offers, you'll find yourself reaching for it constantly.

This guide covers Redis from the ground up: what it is, every major data structure, practical use cases with Node.js code, persistence strategies, clustering, common mistakes, and the best practices that keep Redis running smoothly in production.

What Is Redis?

Redis (Remote Dictionary Server) is an open-source, in-memory data structure store. It was created by Salvatore Sanfilippo in 2009 and has since become one of the most widely used infrastructure tools in modern software development.

The key insight behind Redis is simple: RAM is fast. By keeping all data in memory, Redis delivers response times measured in microseconds rather than the milliseconds typical of disk-based databases. A single Redis instance can handle 100,000+ operations per second without breaking a sweat.

But calling Redis "just a key-value store" misses the point. It's a data structure server. Unlike Memcached (which only stores strings), Redis natively supports strings, hashes, lists, sets, sorted sets, bitmaps, hyperloglogs, and streams. Each structure has its own set of atomic operations — which means you can build complex functionality without external coordination.

Why Redis Is So Popular

  • Speed: All operations happen in memory — sub-millisecond latency for reads and writes
  • Rich data structures: Not just GET/SET — you get sorted sets, streams, geo-spatial indexes, and more
  • Atomic operations: INCR, LPUSH, SADD are all atomic — no race conditions
  • Persistence: Data survives restarts via RDB snapshots and AOF logs
  • Pub/Sub: Built-in publish/subscribe messaging
  • Clustering: Horizontal scaling with automatic sharding
  • Lua scripting: Execute complex logic server-side in a single atomic step
  • Ecosystem: Client libraries for every language, managed hosting on every cloud provider

Redis Data Structures Explained

Each Redis data structure solves a different class of problems. Understanding when to use which one is the key to getting the most out of Redis. Here's every major structure with its commands and real-world applications.

Strings

The simplest data type — a key mapped to a binary-safe string (up to 512MB). But "string" is misleading — Redis strings can store text, serialized JSON, integers, or raw bytes.

Key commands: SET, GET, INCR, DECR, INCRBY, MSET, MGET, SETNX, SETEX, APPEND

Use cases:

  • Caching API responses or database query results
  • Atomic counters (page views, API calls, rate limits)
  • Storing serialized objects (JSON user profiles)
  • Distributed locks with SETNX (SET if Not eXists)
  • Feature flags (simple on/off values)
SET user:1001:name "Alice"           # Store a string
GET user:1001:name                    # → "Alice"
INCR page:views:homepage              # Atomic increment → 1
SETEX cache:products 3600 "[...]"     # Set with 1-hour TTL
MSET key1 "val1" key2 "val2"          # Set multiple keys at once

Hashes

A hash is a map of field-value pairs stored under a single key — like a lightweight object or row. Perfect for representing entities where you need to read or update individual fields without fetching the entire object.

Key commands: HSET, HGET, HMSET, HMGET, HGETALL, HINCRBY, HDEL, HEXISTS, HKEYS, HVALS

Use cases:

  • User profiles (name, email, plan, last_login)
  • Shopping cart items (product_id → quantity)
  • Configuration objects with individual field updates
  • Session data with multiple attributes
  • Rate limit metadata (count, window_start, tier)
HSET user:1001 name "Alice" email "alice@dev.io" plan "pro"
HGET user:1001 email                  # → "alice@dev.io"
HINCRBY user:1001 login_count 1       # Atomic field increment
HGETALL user:1001                     # Get all fields and values
HDEL user:1001 temp_token             # Delete a single field

Lists

Ordered collections of strings implemented as linked lists. O(1) push/pop at both ends, making them ideal for queues and stacks. O(n) for random access by index — don't use them if you need frequent lookups by position.

Key commands: LPUSH, RPUSH, LPOP, RPOP, LRANGE, LLEN, LTRIM, BLPOP, BRPOP, LINDEX

Use cases:

  • Job queues (RPUSH to enqueue, BLPOP to dequeue with blocking)
  • Recent activity feeds (LPUSH new items, LTRIM to cap length)
  • Chat message history (ordered by insertion time)
  • Undo/redo stacks (LPUSH/LPOP)
  • Notification lists (latest N notifications)
RPUSH queue:emails "msg1" "msg2"      # Add to end of queue
BLPOP queue:emails 30                 # Block until item available (30s timeout)
LPUSH feed:user:1001 "posted photo"   # Add to front of activity feed
LTRIM feed:user:1001 0 99             # Keep only latest 100 items
LRANGE feed:user:1001 0 9             # Get 10 most recent items

Sets

Unordered collections of unique strings. Sets shine when you need membership checks, deduplication, and set operations (union, intersection, difference) across multiple sets.

Key commands: SADD, SREM, SMEMBERS, SISMEMBER, SCARD, SUNION, SINTER, SDIFF, SRANDMEMBER, SPOP

Use cases:

  • Tracking unique visitors (SADD with user ID)
  • Online user presence (who's currently active)
  • Tags and categories (articles tagged with "redis")
  • Social features (mutual friends via SINTER)
  • IP allowlists/blocklists (fast SISMEMBER checks)
SADD online:users "user:1001" "user:1002"   # Track online users
SISMEMBER online:users "user:1001"           # → 1 (true)
SCARD online:users                           # → 2 (count)
SADD friends:alice "bob" "carol" "dave"
SADD friends:bob "alice" "carol" "eve"
SINTER friends:alice friends:bob             # → "carol" (mutual friends)

Sorted Sets

Like sets, but every member has a score (floating-point number). Members are automatically ordered by score. This is Redis's most powerful and versatile data structure — think of it as a real-time index you can query by range, rank, or score.

Key commands: ZADD, ZREM, ZSCORE, ZRANK, ZRANGE, ZREVRANGE, ZRANGEBYSCORE, ZINCRBY, ZCARD, ZCOUNT

Use cases:

  • Leaderboards (score = points, member = player)
  • Priority queues (score = priority or timestamp)
  • Rate limiting with sliding windows (score = timestamp)
  • Autocomplete suggestions (score = popularity/frequency)
  • Scheduled jobs (score = execution timestamp)
  • Trending content (score = engagement metric)
ZADD leaderboard 2500 "alice" 1800 "bob" 3100 "carol"
ZREVRANGE leaderboard 0 2 WITHSCORES   # Top 3 players with scores
ZRANK leaderboard "bob"                 # → 0 (lowest rank, 0-indexed)
ZINCRBY leaderboard 500 "bob"           # Add 500 points to bob
ZRANGEBYSCORE leaderboard 2000 3000     # Players with 2000-3000 points
ZCOUNT leaderboard "-inf" "+inf"        # Total players

Streams

Append-only log data structures introduced in Redis 5.0. Streams are designed for event sourcing, message queues, and real-time data processing. They support consumer groups (like Kafka) — multiple consumers can read from the same stream without duplicating work.

Key commands: XADD, XREAD, XRANGE, XLEN, XGROUP CREATE, XREADGROUP, XACK, XTRIM, XINFO

Use cases:

  • Event logs (user actions, system events, audit trails)
  • Message queues with acknowledgment (reliable delivery)
  • Real-time data pipelines (sensor data, click streams)
  • Activity streams with consumer groups
  • Change data capture (CDC) between services
XADD events:orders * action "created" order_id "ord_123" total "49.99"
XADD events:orders * action "paid" order_id "ord_123"
XRANGE events:orders - +                # Read all entries
XLEN events:orders                      # Count entries

# Consumer groups (Kafka-like)
XGROUP CREATE events:orders processors $ MKSTREAM
XREADGROUP GROUP processors worker1 COUNT 10 BLOCK 5000 STREAMS events:orders >
XACK events:orders processors "1656352834-0"   # Acknowledge processed

Redis Use Cases

Here's a quick reference showing the most common Redis use cases, which data structure to use, and why Redis excels at each one.

Use CaseData StructureWhy Redis
CachingStrings / HashesSub-ms reads, automatic TTL expiration, LRU eviction
Session StorageHashesFast field-level access, TTL for expiry, shared across servers
Rate LimitingStrings / Sorted SetsAtomic INCR, sliding windows with ZRANGEBYSCORE, no race conditions
LeaderboardsSorted SetsO(log N) updates, real-time ranking with ZREVRANGE
Pub/Sub MessagingPub/Sub / StreamsBuilt-in fan-out, pattern subscriptions, low latency
Job QueuesLists / StreamsBlocking pop (BLPOP), consumer groups, reliable delivery
Real-time AnalyticsHyperLogLog / Sorted SetsCardinality counting, time-series with sorted sets

Redis in Node.js — Practical Code Examples (ioredis)

The ioredis library is the most popular Redis client for Node.js. It supports clustering, Sentinel, Lua scripting, and pipelining out of the box. Here are four production-ready patterns.

Connection Setup

import Redis from 'ioredis';

const redis = new Redis({
  host: process.env.REDIS_HOST || '127.0.0.1',
  port: 6379,
  password: process.env.REDIS_PASSWORD,
  maxRetriesPerRequest: 3,
  retryStrategy(times) {
    const delay = Math.min(times * 50, 2000);
    return delay; // Reconnect after delay (ms)
  },
  lazyConnect: true,
});

redis.on('error', (err) => console.error('Redis connection error:', err));
redis.on('connect', () => console.log('Redis connected'));

await redis.connect();

1. Cache-Aside Pattern

The most common caching strategy: check Redis first, fall back to the database on cache miss, then populate the cache for future requests.

async function getUser(userId) {
  const cacheKey = `user:${userId}`;

  // 1. Check cache first
  const cached = await redis.get(cacheKey);
  if (cached) {
    return JSON.parse(cached); // Cache hit
  }

  // 2. Cache miss — fetch from database
  const user = await db.users.findById(userId);
  if (!user) return null;

  // 3. Populate cache with 1-hour TTL
  await redis.setex(cacheKey, 3600, JSON.stringify(user));

  return user;
}

// Invalidate on update
async function updateUser(userId, data) {
  await db.users.update(userId, data);
  await redis.del(`user:${userId}`); // Invalidate cache
}

2. Session Store

Store user sessions in Redis hashes. Each session gets its own key with a TTL. Individual fields can be read or updated without fetching the entire session.

import { randomUUID } from 'crypto';

async function createSession(userId, metadata = {}) {
  const sessionId = randomUUID();
  const sessionKey = `session:${sessionId}`;
  const TTL = 86400; // 24 hours

  await redis.hmset(sessionKey, {
    userId,
    createdAt: Date.now().toString(),
    userAgent: metadata.userAgent || '',
    ip: metadata.ip || '',
  });
  await redis.expire(sessionKey, TTL);

  return sessionId;
}

async function getSession(sessionId) {
  const data = await redis.hgetall(`session:${sessionId}`);
  if (!data || !data.userId) return null;
  return data;
}

async function destroySession(sessionId) {
  await redis.del(`session:${sessionId}`);
}

// Extend session on activity
async function touchSession(sessionId) {
  await redis.expire(`session:${sessionId}`, 86400);
}

3. Sliding Window Rate Limiter

Using sorted sets for a precise sliding window. Each request is stored with a timestamp score. Old entries are pruned, and the remaining count determines if the request is allowed.

async function isRateLimited(clientId, limit = 100, windowSec = 60) {
  const key = `ratelimit:${clientId}`;
  const now = Date.now();
  const windowStart = now - windowSec * 1000;

  const pipeline = redis.pipeline();
  pipeline.zremrangebyscore(key, 0, windowStart); // Remove old entries
  pipeline.zadd(key, now, `${now}:${Math.random()}`); // Add current request
  pipeline.zcard(key); // Count requests in window
  pipeline.expire(key, windowSec); // Set TTL for cleanup

  const results = await pipeline.exec();
  const requestCount = results[2][1];

  if (requestCount > limit) {
    // Remove the entry we just added (request denied)
    await redis.zrem(key, `${now}:${Math.random()}`);
    return { limited: true, remaining: 0, retryAfter: windowSec };
  }

  return { limited: false, remaining: limit - requestCount };
}

4. Pub/Sub Pattern

Redis pub/sub enables real-time messaging between services. Note that pub/sub requires a separate Redis connection for the subscriber (you can't mix pub/sub with regular commands on the same connection).

// Publisher (any service)
async function publishEvent(channel, event) {
  await redis.publish(channel, JSON.stringify({
    ...event,
    timestamp: Date.now(),
  }));
}

// Subscriber (separate connection required)
const subscriber = new Redis({ host: process.env.REDIS_HOST });

subscriber.subscribe('orders:created', 'orders:paid', (err, count) => {
  console.log(`Subscribed to ${count} channels`);
});

subscriber.on('message', (channel, message) => {
  const event = JSON.parse(message);
  switch (channel) {
    case 'orders:created':
      sendOrderConfirmationEmail(event);
      break;
    case 'orders:paid':
      triggerFulfillment(event);
      break;
  }
});

// Pattern subscription (wildcard)
subscriber.psubscribe('orders:*');
subscriber.on('pmessage', (pattern, channel, message) => {
  logToAnalytics(channel, JSON.parse(message));
});

Redis vs Memcached

Memcached was the original in-memory cache (2003). Redis came later (2009) and offers significantly more features. Here's when each one makes sense.

FeatureRedisMemcached
Data StructuresStrings, Hashes, Lists, Sets, Sorted Sets, Streams, BitmapsStrings only
PersistenceRDB snapshots + AOF logsNone (volatile only)
ReplicationBuilt-in primary/replica replicationNone (use external tools)
Pub/SubNative pub/sub + StreamsNot supported
ScriptingLua scripting (atomic execution)Not supported
ClusteringRedis Cluster with auto-shardingClient-side sharding only
ThreadingSingle-threaded commands, multi-threaded I/O (v6+)Multi-threaded
Max Value Size512 MB1 MB (default)
Memory EfficiencyGood (with ziplist encoding for small structures)Slightly better for simple strings
Best ForComplex use cases, persistence needed, rich data typesSimple caching, very high connection counts

Bottom line: Choose Redis unless you have a specific reason to use Memcached (legacy infrastructure, extremely high connection counts with simple string caching). Redis does everything Memcached does and much more.

Redis Persistence — RDB vs AOF

Redis stores data in memory, but that doesn't mean it disappears on restart. Redis offers two persistence mechanisms that can be used independently or together.

RDB (Redis Database Backup)

RDB creates point-in-time snapshots of your entire dataset at configured intervals. The snapshot is saved as a compact binary file (dump.rdb) that Redis can load on startup.

  • Compact single-file backups — easy to copy to remote storage
  • Faster restarts for large datasets (binary format loads quickly)
  • Minimal performance impact (child process does the work via fork)
  • Risk: data written between snapshots is lost on crash

AOF (Append-Only File)

AOF logs every write operation. On restart, Redis replays the log to rebuild the dataset. You can configure how often the log is fsynced to disk.

  • Better durability — can be configured to fsync every second or every write
  • Append-only = no corruption from partial writes
  • Human-readable log (useful for debugging)
  • AOF rewrite compacts the log periodically (removes redundant operations)
  • Larger file size than RDB, slower restarts for very large datasets

Comparison

AspectRDBAOF
DurabilityMinutes of data loss possibleAt most 1 second of data loss
File SizeCompact (binary)Larger (text log, compacted periodically)
Restart SpeedFast (binary load)Slower (replay operations)
Performance ImpactLow (fork + background write)Higher (fsync overhead)
Backup FriendlyExcellent (single file, copy anywhere)Good (but larger files)
RecoveryAll-or-nothing (last snapshot)Can truncate corrupted tail

Recommendation: Use both in production. RDB for fast restarts and backups, AOF for durability. If you must choose one, AOF withappendfsync everysec gives a good balance between performance and data safety.

Redis Cluster & Sentinel — Scaling and High Availability

A single Redis instance is fast, but it has limits: memory capacity of one machine, and a single point of failure. Redis provides two solutions for production environments.

Redis Sentinel (High Availability)

Sentinel monitors your Redis instances and automatically promotes a replica to primary if the current primary fails. It handles:

  • Monitoring: Continuously checks if primary and replicas are working
  • Notification: Alerts administrators via API when something goes wrong
  • Automatic failover: Promotes a replica to primary when the primary is unreachable
  • Configuration provider: Clients ask Sentinel for the current primary address

Use Sentinel when your dataset fits in a single machine's memory but you need automatic failover. A typical setup is one primary, two replicas, and three Sentinel processes (running on separate machines for quorum decisions).

Redis Cluster (Horizontal Scaling)

Redis Cluster automatically shards data across multiple nodes using 16,384 hash slots. Each key is assigned to a slot (via CRC16), and each node owns a subset of slots.

  • Automatic sharding: Data distributed across nodes without application logic
  • Linear scalability: Add nodes to increase capacity and throughput
  • Built-in failover: Each primary has replicas; promotion happens automatically
  • Multi-key limitations: Operations spanning multiple keys must use the same hash slot (use hash tags)

Use Cluster when a single instance can't hold all your data or when you need more write throughput than one node provides. Start with 6 nodes minimum (3 primaries + 3 replicas).

ioredis Cluster Connection

import Redis from 'ioredis';

const cluster = new Redis.Cluster([
  { host: 'redis-node-1', port: 6379 },
  { host: 'redis-node-2', port: 6379 },
  { host: 'redis-node-3', port: 6379 },
], {
  redisOptions: { password: process.env.REDIS_PASSWORD },
  scaleReads: 'slave', // Read from replicas for read-heavy workloads
  natMap: {}, // Use if behind NAT/Docker
});

// Usage is identical to single-instance — ioredis handles routing
await cluster.set('key', 'value');
const val = await cluster.get('key');

Common Redis Mistakes

These are the mistakes I see most often in production Redis deployments. Each one can cause outages, data loss, or performance degradation if left unchecked.

1. Using Redis as Your Primary Database

Redis is excellent as a complement to your primary database — not a replacement. RAM is expensive, and Redis doesn't support complex queries, joins, or ACID transactions the way PostgreSQL or MySQL do. Use Redis for specific workloads (caching, sessions, counters) and a proper database for your source of truth.

2. Not Setting TTL on Keys

Keys without a TTL live forever. Over time, your Redis instance fills up with stale data until you hit maxmemory and eviction kicks in (often evicting keys you still need). Set explicit TTLs on cache entries and temporary data. Audit keyspace regularly for orphaned keys.

3. Storing Huge Values

Storing 10MB+ JSON blobs in a single key blocks the event loop during serialization and transfer. Redis is single-threaded for commands — one slow operation blocks everything. Keep values small (under 100KB ideally). Split large objects into hashes or multiple keys. Use compression for larger payloads.

4. Not Handling Connection Failures

Redis will go down eventually — network blips, maintenance, OOM kills. If your application crashes or hangs when Redis is unavailable, you have a reliability problem. Implement circuit breakers, connection timeouts, retry logic with backoff, and graceful degradation (serve from database if cache is unavailable).

5. Using KEYS Command in Production

KEYS * scans the entire keyspace and blocks Redis until complete. With millions of keys, this takes seconds — blocking all other clients. Use SCAN instead (cursor-based, non-blocking iteration). Disable KEYS in production via rename-command configuration.

6. Ignoring Memory Management

Redis uses more memory than the raw data size (pointers, metadata, encoding overhead). If you don't set maxmemory and an eviction policy, Redis grows until the OS kills it (OOM). Always configure maxmemory, choose an appropriate eviction policy (allkeys-lru for caches), and monitor memory usage with alerts.

Redis Best Practices

Follow these practices to keep Redis reliable, performant, and maintainable in production.

1. Use Meaningful Key Naming Conventions

Adopt a consistent pattern like object-type:id:field (e.g., user:1001:profile, cache:products:list). Use colons as separators. Prefix with environment if sharing instances (prod:user:1001). This makes debugging and key scanning much easier.

2. Always Set maxmemory and Eviction Policy

Configure maxmemory to 70-80% of available RAM (leave room for fork operations during persistence). Use allkeys-lru for caches, volatile-lru if you mix persistent and cached data. Never run Redis without a memory limit in production.

3. Use Pipelining for Bulk Operations

Each Redis command requires a network round-trip. If you're executing 100 commands sequentially, that's 100 round-trips. Pipelining batches commands into a single round-trip, dramatically improving throughput (10x+ for bulk operations).

4. Monitor with INFO and SLOWLOG

Run INFO stats regularly to track memory usage, connected clients, keyspace hits/misses, and replication lag. Enable SLOWLOG to catch commands taking longer than expected. Set up alerts for memory > 80%, hit rate < 90%, and replication lag > 1 second.

5. Use Connection Pooling

Don't create a new Redis connection per request. Use a connection pool (ioredis handles this automatically). A pool of 10-50 connections is typically sufficient for most applications. Monitor pool exhaustion as a sign you need more capacity.

6. Implement Cache Stampede Protection

When a popular cache key expires, hundreds of requests simultaneously hit the database (thundering herd). Protect against this with mutex locks (SETNX), probabilistic early expiration, or background cache refresh before TTL expires.

7. Use Lua Scripts for Atomic Multi-Step Operations

If your logic requires read-then-write (check a value, then update based on it), use a Lua script to execute atomically. This eliminates race conditions without external locking. Redis guarantees Lua scripts run without interruption.

8. Separate Concerns with Multiple Redis Instances

Don't use one Redis instance for everything. Run separate instances (or databases) for cache, sessions, and queues. A cache eviction should never accidentally delete session data. Different workloads have different persistence and eviction requirements.

Frequently Asked Questions

What is Redis and why do developers use it?

Redis is an open-source, in-memory data structure store used as a database, cache, message broker, and streaming engine. Developers use it for sub-millisecond response times, rich data structures, atomic operations, and versatility — it handles caching, sessions, rate limiting, queues, pub/sub, and real-time analytics.

Is Redis just a cache?

No. While caching is its most common use case, Redis is a full data structure server with persistence, pub/sub messaging, streams, Lua scripting, transactions, and clustering. Many teams use it as a primary store for sessions, leaderboards, and real-time counters.

What is the difference between Redis and Memcached?

Redis supports rich data structures, persistence, replication, scripting, and pub/sub. Memcached only supports simple key-value strings with no persistence. Redis is the better choice for most use cases unless you specifically need Memcached's multi-threaded architecture for simple string caching at extremely high connection counts.

How does Redis persistence work?

Redis offers RDB (periodic snapshots) and AOF (append-only log of every write). RDB is great for backups and fast restarts. AOF provides better durability (at most 1 second of data loss). Most production deployments use both together.

Can Redis handle millions of requests per second?

A single Redis instance handles 100,000+ ops/sec. With Redis Cluster sharding across multiple nodes, you can achieve millions of operations per second. Redis 6+ multi-threaded I/O further improves throughput for network-bound workloads.

Should I use Redis as my primary database?

Generally no. RAM is expensive for large datasets, and Redis lacks complex queries and joins. Use Redis alongside your primary database for specific workloads: caching, sessions, rate limiting, counters, and pub/sub. Some teams use it as primary for small, performance-critical datasets.

What is Redis Cluster and when do I need it?

Redis Cluster shards data across multiple nodes using 16,384 hash slots. You need it when a single instance can't hold all your data or when you need more write throughput. For most applications under 25GB, a single instance with Sentinel for failover is sufficient.

How do I handle Redis connection failures in production?

Implement connection pooling, automatic reconnection with exponential backoff, circuit breakers, and graceful degradation. Your application should continue working (possibly slower) if Redis is unavailable — fall back to database queries or serve stale data.

Related Articles

Conclusion

Redis is one of those tools that keeps revealing new capabilities the more you use it. What starts as "I need a cache" quickly becomes "I'm using sorted sets for leaderboards, streams for event processing, and pub/sub for real-time notifications."

The key takeaways:

  • Redis is a data structure server, not just a cache — learn the structures and you'll find uses everywhere
  • Choose the right data structure for each use case (sorted sets for ranking, hashes for objects, streams for events)
  • Always set TTLs, configure maxmemory, and handle connection failures gracefully
  • Use both RDB and AOF persistence in production for the best balance of speed and durability
  • Start with a single instance + Sentinel; add Cluster only when you genuinely need horizontal scaling
  • Separate workloads (cache vs sessions vs queues) across different Redis instances

Redis isn't going anywhere. It's battle-tested across millions of production deployments, from small startups to the largest tech companies. Understanding it deeply will serve you well throughout your career as a developer.