API Gateway Explained

You start with one backend service. Then two. Then twelve. Suddenly your mobile app is making six different API calls just to render a dashboard, your web client has auth logic duplicated everywhere, and each service has its own rate limiting that nobody configured the same way. Sound familiar? That's the problem an API gateway solves.

This guide covers what an API gateway actually is (without the marketing fluff), how it works under the hood, the core features you should care about, a hands-on code example, popular options compared, and — just as importantly — when you should skip it entirely.

What Is an API Gateway?

An API gateway is a server that sits between your clients (web apps, mobile apps, third-party integrations) and your backend services. It acts as a single entry point for all incoming requests. Instead of clients knowing about and communicating directly with ten different microservices, they talk to one gateway — and the gateway figures out where to route each request.

In simpler terms: it's a reverse proxy that also handles cross-cutting concerns. Routing, authentication, rate limiting, request transformation, logging, caching — all of these happen at the gateway level so your individual services don't have to worry about them.

The concept isn't new. It's essentially the Facade pattern applied to distributed systems. You present a clean, unified interface to the outside world while hiding the complexity of what's happening behind the curtain.

Why Use an API Gateway?

The short answer: because managing cross-cutting concerns across dozens of services individually is a nightmare. Here's what centralizing through a gateway gives you:

Centralized Authentication

Without a gateway, every service needs to validate JWT tokens, check API keys, or verify OAuth flows. That means duplicated auth code across every service, in potentially different languages. With a gateway, auth happens once at the edge. If the token is valid, the request moves forward. If not, it gets rejected before touching any backend service.

Rate Limiting

You want to limit a client to 100 requests per minute across your entire platform — not 100 per service. A gateway gives you a single enforcement point with a unified view of each client's request volume.

Unified Logging & Monitoring

Every request flows through the gateway, making it the natural place to log request metadata, measure latency, track error rates, and feed data into your observability stack. You get a complete picture of your API traffic without instrumenting each service separately.

Request/Response Transformation

Your mobile app needs a slimmed-down response. A legacy client expects XML. A new partner wants fields renamed. The gateway can transform requests and responses on the fly without touching service code.

Service Abstraction

Clients don't need to know (or care) about your internal service topology. You can split a service into three, merge two services into one, or migrate from Node.js to Go — the client sees the same API contract through the gateway.

How an API Gateway Works

The flow is straightforward. A client makes a request. The gateway intercepts it, applies policies, routes it to the right backend service (or multiple services), collects the response, optionally transforms it, and sends it back to the client.

Here's the flow visualized:

┌──────────────┐         ┌─────────────────────────────────┐         ┌──────────────────┐
│              │         │         API GATEWAY              │         │                  │
│   Client     │────────▶│                                 │────────▶│  User Service    │
│  (Web/Mobile)│         │  1. Receive request             │         │  /api/users/*    │
│              │◀────────│  2. Authenticate (JWT/API Key)  │         └──────────────────┘
└──────────────┘         │  3. Rate limit check            │
                         │  4. Route to service            │         ┌──────────────────┐
                         │  5. Transform response          │────────▶│  Order Service   │
                         │  6. Log & monitor               │         │  /api/orders/*   │
                         │  7. Return to client            │         └──────────────────┘
                         │                                 │
                         │                                 │         ┌──────────────────┐
                         │                                 │────────▶│  Payment Service │
                         │                                 │         │  /api/payments/* │
                         └─────────────────────────────────┘         └──────────────────┘

The key insight: the client makes one request to one endpoint. The gateway might fan that out to three services behind the scenes, aggregate the responses, and return a single payload. The client never knows (or cares) about the complexity behind the gateway.

API Gateway vs Direct Client-to-Service Communication

Not every architecture needs a gateway. Here's how the two approaches compare:

The takeaway: gateways shine when you have multiple services and shared concerns. Direct communication is simpler when you have few services or need minimal latency.

Core Features of an API Gateway

Not every gateway offers every feature, but here are the capabilities that define the category:

Popular API Gateways Compared

There's no single "best" gateway. The right choice depends on your infrastructure, team skills, and requirements. Here's how the major options stack up:

Code Example: Building a Simple API Gateway with Node.js

You don't always need Kong or AWS. For smaller setups or learning purposes, you can build a basic API gateway with Express and http-proxy-middleware. Here's a working example that routes requests to different microservices:

// gateway.js — A minimal API gateway with Express
const express = require('express');
const { createProxyMiddleware } = require('http-proxy-middleware');
const rateLimit = require('express-rate-limit');

const app = express();

// ─── Middleware: Rate limiting (global) ───────────────────
const limiter = rateLimit({
  windowMs: 60 * 1000,  // 1 minute
  max: 100,             // 100 requests per minute per IP
  message: { error: 'Too many requests. Try again in a minute.' },
});
app.use(limiter);

// ─── Middleware: Simple API key auth ──────────────────────
function authenticate(req, res, next) {
  const apiKey = req.headers['x-api-key'];
  if (!apiKey || apiKey !== process.env.API_KEY) {
    return res.status(401).json({ error: 'Invalid or missing API key' });
  }
  next();
}
app.use(authenticate);

// ─── Middleware: Request logging ──────────────────────────
app.use((req, res, next) => {
  const start = Date.now();
  res.on('finish', () => {
    console.log(`[${req.method}] ${req.path} → ${res.statusCode} (${Date.now() - start}ms)`);
  });
  next();
});

// ─── Route: User Service ──────────────────────────────────
app.use('/api/users', createProxyMiddleware({
  target: 'http://localhost:3001',
  changeOrigin: true,
  pathRewrite: { '^/api/users': '/users' },
}));

// ─── Route: Order Service ─────────────────────────────────
app.use('/api/orders', createProxyMiddleware({
  target: 'http://localhost:3002',
  changeOrigin: true,
  pathRewrite: { '^/api/orders': '/orders' },
}));

// ─── Route: Payment Service ───────────────────────────────
app.use('/api/payments', createProxyMiddleware({
  target: 'http://localhost:3003',
  changeOrigin: true,
  pathRewrite: { '^/api/payments': '/payments' },
}));

// ─── Health check (no auth required) ─────────────────────
app.get('/health', (req, res) => res.json({ status: 'ok', uptime: process.uptime() }));

// ─── Start gateway ───────────────────────────────────────
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`API Gateway running on port ${PORT}`));

This gives you routing, rate limiting, authentication, and logging in about 60 lines. In production you'd add circuit breaking (with opossum), caching (with apicache), and health checks for upstream services. But this shows the core pattern clearly.

API Gateway Patterns

Beyond basic routing, there are three architectural patterns that use gateways in different ways:

1. BFF — Backend for Frontend

Instead of one gateway serving all clients the same way, you create a dedicated gateway (or gateway configuration) per client type. Your mobile app gets a BFF that returns minimal payloads optimized for bandwidth. Your web app gets a BFF that returns richer data. A third-party integration gets a BFF with strict rate limits and different auth.

This avoids the "one API fits all" problem where your mobile app downloads 50 fields when it only needs 5, or your web app makes three calls to assemble what could be one response.

┌─────────────┐       ┌──────────────────┐
│  Mobile App │──────▶│  Mobile BFF      │──┐
└─────────────┘       └──────────────────┘  │    ┌──────────────┐
                                            ├───▶│  Services    │
┌─────────────┐       ┌──────────────────┐  │    └──────────────┘
│  Web App    │──────▶│  Web BFF         │──┘
└─────────────┘       └──────────────────┘

┌─────────────┐       ┌──────────────────┐       ┌──────────────┐
│  Partner    │──────▶│  Partner BFF     │──────▶│  Services    │
└─────────────┘       └──────────────────┘       └──────────────┘

2. Gateway Aggregation

The gateway collects data from multiple backend services, combines them into a single response, and returns it to the client. Classic example: a dashboard page that needs user profile data from the User Service, recent orders from the Order Service, and notification count from the Notification Service. Without aggregation, the client makes three separate API calls.

With aggregation, the client makes one call. The gateway fans out to three services in parallel, waits for all responses, merges them, and returns one JSON object. Fewer round trips, less client-side complexity, better perceived performance.

3. Gateway Offloading

Move shared functionality from services into the gateway. SSL termination, compression, CORS headers, request validation, IP whitelisting — these concerns apply to all services identically. Instead of implementing them in each service, offload them to the gateway.

This keeps your services focused on business logic rather than infrastructure concerns. The gateway handles the "boilerplate" — services handle the "business."

When NOT to Use an API Gateway

Gateways aren't free. They add latency, complexity, and another thing to maintain. Here are situations where you're better off without one:

Common Mistakes When Implementing an API Gateway

I've seen these trip up teams repeatedly. Avoid them and you'll save yourself production incidents:

1. Putting business logic in the gateway. The gateway should handle routing, auth, and policies — not domain-specific validation, complex data transformations, or business rules. The moment your gateway has "if product category is X, apply discount Y" logic, you've created a God object that's impossible to test.
2. No health checks for upstream services. If the gateway blindly forwards requests to a downed service, clients get timeouts. Implement active health checks and remove unhealthy instances from the routing table automatically.
3. Single instance without redundancy. The gateway is your single entry point — which means it's a single point of failure. Run multiple instances behind a load balancer with auto-scaling. If the gateway dies, everything dies.
4. Forgetting about timeouts. If a backend service hangs, the gateway holds the connection open, consuming resources. Set aggressive timeouts (5-15 seconds) and implement circuit breakers to fail fast rather than wait indefinitely.
5. Over-engineering for scale you don't have. Don't deploy Kong with PostgreSQL, Redis, and Consul for three services getting 100 requests per minute. Start simple. A basic Nginx config or a small Express proxy handles thousands of requests per second just fine.
6. Ignoring observability. If you can't see what's happening inside the gateway — request latency, error rates, which services are slow — you're flying blind. Configure structured logging, metrics export (Prometheus), and distributed tracing (OpenTelemetry) from day one.

API Gateway Best Practices

Follow these guidelines to keep your gateway reliable, maintainable, and performant:

1. Keep the gateway thin. Route, authenticate, rate limit, log — that's it. No business logic. The gateway is infrastructure, not application code. If you find yourself writing domain-specific conditionals, push that logic into the services where it belongs.
2. Version your gateway configuration. Treat gateway config (routes, policies, rate limits) as code. Store it in Git, review changes via pull requests, and deploy it through CI/CD. "I changed the route in the admin UI" leads to mystery outages with no audit trail.
3. Implement circuit breakers for all upstream services. When a service is failing, stop hammering it. Open the circuit, return a cached response or a graceful error, and check periodically if the service has recovered. Libraries like opossum (Node.js) or resilience4j (Java) make this straightforward.
4. Use connection pooling. Don't open a new TCP connection for every proxied request. Maintain connection pools to backend services. This reduces latency significantly (avoiding TCP handshakes and TLS negotiation on every request).
5. Add request IDs for tracing. Generate a unique request ID at the gateway (or preserve one from the client) and propagate it to all downstream services. When something fails, you can trace the entire request flow across services using that ID.
6. Set appropriate timeouts at every level. Client → gateway timeout, gateway → service timeout, and service → database timeout should each be progressively shorter. A 30-second client timeout with a 60-second gateway timeout means the gateway holds dead connections.
7. Plan for zero-downtime deployments. Blue-green or rolling deployments for the gateway ensure you never take down all API traffic while updating routes or configuration. Test new configurations against canary traffic before full rollout.
8. Monitor gateway-specific metrics. Track: requests per second, p50/p95/p99 latency, error rate by upstream service, active connections, rate limit hits, and circuit breaker state. Set alerts on anomalies — not just thresholds.

Frequently Asked Questions

Related Articles & Tools

Keep building your API knowledge with these related resources:

Conclusion

An API gateway is one of those architectural components that seems like overkill — right up until the moment you desperately need one. When you have multiple services, multiple client types, and cross-cutting concerns like auth and rate limiting that need to be consistent across all of them, a gateway transforms chaos into something manageable.

But it's not a universal answer. For monoliths, simple apps, or early-stage projects, a gateway adds complexity you don't need yet. The key is recognizing when the pain of managing distributed concerns manually exceeds the cost of adding and maintaining a gateway.

Start simple. A basic reverse proxy with routing covers a lot of ground. Add features (auth, rate limiting, circuit breaking) as you actually need them. And whatever you do — keep business logic out of the gateway. Let it be infrastructure. Let your services be the brains.