APIs & Communication: Wrap-Up
Systems Design
| # | Post | What it covers |
|---|---|---|
| 00 | APIs & Communication: How Services Talk to Each Other | How services talk to each other shapes everything about a system. Nine concepts covering REST, WebSockets, async patterns, and API gateways. (146 chars) |
| 01 | API Design: Building Contracts That Last | A great API is a contract that outlasts your code. Here are the principles that make APIs intuitive to consume, safe to evolve, and cheap to maintain. (154 chars) |
| 02 | REST APIs: Constraints That Create Benefits | REST isn't just HTTP with JSON. It's an architectural style with specific constraints — and understanding them explains why REST APIs are designed the way they are. (166 chars) |
| 03 | Authentication vs Authorisation: Two Questions, Two Checks | Authentication is who you are. Authorisation is what you're allowed to do. Confusing them is one of the most common security mistakes in system design. (153 chars) |
| 04 | Session vs Token Authentication: Stateful vs Stateless Identity | Session auth stores identity on the server. Token auth encodes it in the token. Here's how each works, where each breaks, and how to choose. (144 chars) |
| 05 | OAuth 2.0 & OpenID Connect: Delegated Access and Federated Identity | OAuth 2.0 lets users grant apps access without sharing passwords. OpenID Connect adds identity on top. Here's how both actually work. (137 chars) |
| 06 | JWT: What's Actually Inside the Token | JWTs are everywhere in modern auth — and frequently misused. Here's exactly what a JWT contains, how the signature works, and what it doesn't protect. (153 chars) |
| 07 | WebSockets: Real-Time Bidirectional Communication | HTTP is request-response. WebSockets are a persistent two-way channel. Here's how they work, when to use them, and what to watch out for at scale. (151 chars) |
| 08 | Long Polling, SSE & Webhooks: The Server-Push Spectrum | Three patterns for server-push communication — long polling, server-sent events, and webhooks. Here's how each works and when to reach for each. (150 chars) |
| 09 | Sync vs Async Communication: The Architectural Fork | Synchronous services couple tightly. Asynchronous services decouple — but add complexity. Here's how to reason about which your system needs. (147 chars) |
| 10 | API Gateways: One Entry Point, Every Cross-Cutting Concern | An API gateway centralises auth, rate limiting, routing, and observability for all your services. Here's what it does, how it works, and when you need one. (158 chars) |
| 11 | APIs & Communication: Wrap-Up ← you are here | A complete recap of all ten API and communication concepts — REST, auth, JWT, WebSockets, webhooks, async patterns, and API gateways — and how they connect. (161 chars) |
APIs & Communication: Wrap-Up
Ten posts. A complete picture of how services communicate with each other and with the outside world. Before moving to the next pillar, it's worth tracing how these concepts connect — because no production system uses just one of them in isolation.
What we covered
API Design
APIs are contracts that outlast your code. Design for the consumer, not the implementation. Consistent naming, predictable error responses, sensible versioning, and idempotency keys for state-changing operations. The investment at design time saves orders of magnitude more time across the lifetime of every integration built against your API.
The one thing to remember: design your API for the engineer debugging a production incident at 2am, not for the engineer implementing it.
REST APIs
REST is an architectural style — six constraints that, when applied, give distributed systems specific properties: scalability, cacheability, modifiability. Most APIs reach Richardson maturity level 2 (resource URLs, correct HTTP methods, correct status codes) and that's sufficient for most practical purposes. Understand the constraints you're applying and deviate deliberately when they don't serve your use case.
The one thing to remember: REST is a set of constraints that create benefits, not a naming convention for HTTP APIs.
Authentication vs Authorisation
Authentication answers "who are you?" Authorisation answers "what are you allowed to do?" They are separate checks, applied in sequence, for different purposes. Function-level authorisation (can this role do this operation?) and object-level authorisation (does this user own this resource?) must both be checked on every request.
The one thing to remember: an authenticated request that bypasses authorisation isn't a secured system — it's a system where every authenticated user can do everything.
Session vs Token Authentication
Sessions store identity on the server (session store lookup on every request, instant revocation). Tokens encode identity in the token (stateless validation, no lookup, revocation requires short expiry plus refresh tokens). For modern API-first architectures with multiple client types, token auth with short-lived access tokens and revocable refresh tokens is standard. For traditional web apps where instant revocation matters more than portability, sessions remain appropriate.
The one thing to remember: the choice is about where state lives and who controls it. Sessions give the server control. Tokens give clients portability.
OAuth 2.0 & OpenID Connect
OAuth solves the credential-sharing problem: users grant limited, scoped, revocable access to third parties without sharing passwords. OIDC adds federated identity on top: authenticate once, use that identity everywhere. Use an identity provider rather than building your own — the security complexity of an authorisation server is not where engineering effort should go.
The one thing to remember: OAuth is for delegated authorisation. OIDC is for federated authentication. They're complementary, not competing.
JWT
A JWT proves a payload was issued by a trusted party and hasn't been tampered with. It doesn't encrypt the payload, can't revoke itself, and is only as secure as your validation code. Always specify allowed algorithms explicitly. Never put sensitive data in the payload. Short expiry plus revocable refresh tokens handles the revocation problem.
The one thing to remember: a JWT's payload is readable by anyone. The signature proves authenticity, not privacy.
WebSockets
A persistent, bidirectional TCP channel where either party can send messages at any time. Right for genuine bidirectional real-time communication — collaborative editing, multiplayer games, live trading. Stateful, operationally complex, requires a pub/sub layer for fan-out at scale. Not the default for server-push — SSE is simpler for that.
The one thing to remember: use WebSockets when both sides need to initiate communication and latency matters. For server-only push, SSE is simpler.
Long Polling, SSE & Webhooks
Three patterns for "the server needs to tell the client something": long polling simulates push with persistent pull; SSE streams events down a single HTTP connection (one direction, auto-reconnect, HTTP/2-friendly); webhooks make the server call the client (no persistent connection, fire-and-forget with retries). SSE is the default for browser clients. Webhooks are the default for server-to-server event delivery.
The one thing to remember: match the pattern to the context — SSE for browser streams, webhooks for server-to-server events, long polling as a fallback.
Sync vs Async Communication
Synchronous couples the caller to every service in the chain — their response time and availability become your response time and availability. Asynchronous decouples them — at the cost of eventual consistency and operational complexity. Use synchronous for operations the user waits for. Use asynchronous for side effects, analytics, notifications, and anything the user doesn't need confirmed immediately.
The one thing to remember: if the user would wait for it, make it synchronous. If they wouldn't notice the lag, make it asynchronous.
API Gateways
A single entry point for all external traffic that handles cross-cutting concerns — authentication, rate limiting, routing, transformation, observability — consistently, for all services. Eliminates per-service reimplementation and drift. Keep it thin (infrastructure concerns only, no business logic), version-control its configuration, and ensure it's highly available.
The one thing to remember: the gateway solves the cross-cutting concern problem. It's not where business logic lives — it's where infrastructure concerns live.
How they connect: the URL shortener platform in full
Tracing a partner's webhook registration and a subsequent link-creation event through every concept in this pillar:
Partner registers webhook endpoint:
POST /webhooks
Authorization: Bearer eyJ... ← JWT (post 06)
{"url": "https://partner.com/events", "events": ["link.created"]}
API Gateway (post 10):
├── Validates JWT signature, expiry, issuer ← Authentication (post 03, 04)
├── Checks scope: webhooks:write ← Authorisation (post 03)
├── Checks rate limit: 10 req/min (partner tier)
└── Routes to webhook-service:8084
webhook-service:
Stores endpoint, returns 201 Created ← REST (post 02), API Design (post 01)
User creates a short link:
POST /links
Authorization: Bearer eyJ...
{"destination": "https://example.com"}
API Gateway: validates, routes to link-service
link-service:
├── Writes link record to database ← Synchronous (post 09)
├── Returns 201 Created to user ← Immediate response
└── Publishes link.created event ← Asynchronous (post 09)
│
├──► Webhook dispatcher consumes event
│ └── POST https://partner.com/events ← Webhook (post 08)
│ {"event": "link.created", ...}
│ Signed with HMAC-SHA256
│
└──► Analytics service consumes event
└── Pushes click_update to dashboard ← SSE (post 08)
via Redis Pub/Sub → WebSocket (post 07)
Partner authenticates their integration:
Uses OAuth 2.0 client credentials flow ← OAuth (post 05)
Receives access token with scope: links:read
Calls GET /links/{id}
API Gateway validates token, checks scope ← API Gateway (post 10)
Every concept in this pillar appears in a real platform doing real work.
What's next: Data & Storage
The APIs pillar covered how services talk to each other. The next pillar covers where data lives and how it scales:
- SQL vs NoSQL — choosing the right database for the data model
- Indexing and query optimisation — making reads fast
- Sharding, partitioning, and consistent hashing — distributing data across nodes
- Replication and read replicas — scaling reads and surviving failures
- Specialised stores — time series, vector databases, full-text search engines
- Data structures — B-trees, LSM trees, bloom filters, and the algorithms that make databases fast
The APIs pillar gave you the communication layer. The Data & Storage pillar gives you the persistence layer — where all the data those APIs create, read, and modify actually lives.
Start Pillar 4 → Data & Storage: Overview
Part of the System Design series. Tags: #systemdesign #api #distributedsystems #backend #softwarearchitecture #engineering
← Previous: API Gateways: One Entry Point, Every Cross-Cutting Concern — An API gateway centralises auth, rate limiting, routing, and observability for all your services. Here's what it does...
