Series: System Design · APIs & Communication — Pillar 3 of 8 This pillar: 00 — Overview · 01 — API Design · 02 — REST APIs · 03 — Authentication vs Authorisation · 04 — Session vs Token Auth · 05 — OAuth & OpenID Connect · 06 — JWT · 07 — WebSockets · 08 — Long Polling, SSE & Webhooks · 09 — Sync vs Async Communication · 10 — API Gateways · 11 — Wrap-up

---

Long Polling, SSE & Webhooks: The Server-Push Spectrum

Systems Design

#	Post	What it covers
00	APIs & Communication: How Services Talk to Each Other	How services talk to each other shapes everything about a system. Nine concepts covering REST, WebSockets, async patterns, and API gateways. (146 chars)
01	API Design: Building Contracts That Last	A great API is a contract that outlasts your code. Here are the principles that make APIs intuitive to consume, safe to evolve, and cheap to maintain. (154 chars)
02	REST APIs: Constraints That Create Benefits	REST isn't just HTTP with JSON. It's an architectural style with specific constraints — and understanding them explains why REST APIs are designed the way they are. (166 chars)
03	Authentication vs Authorisation: Two Questions, Two Checks	Authentication is who you are. Authorisation is what you're allowed to do. Confusing them is one of the most common security mistakes in system design. (153 chars)
04	Session vs Token Authentication: Stateful vs Stateless Identity	Session auth stores identity on the server. Token auth encodes it in the token. Here's how each works, where each breaks, and how to choose. (144 chars)
05	OAuth 2.0 & OpenID Connect: Delegated Access and Federated Identity	OAuth 2.0 lets users grant apps access without sharing passwords. OpenID Connect adds identity on top. Here's how both actually work. (137 chars)
06	JWT: What's Actually Inside the Token	JWTs are everywhere in modern auth — and frequently misused. Here's exactly what a JWT contains, how the signature works, and what it doesn't protect. (153 chars)
07	WebSockets: Real-Time Bidirectional Communication	HTTP is request-response. WebSockets are a persistent two-way channel. Here's how they work, when to use them, and what to watch out for at scale. (151 chars)
08	Long Polling, SSE & Webhooks: The Server-Push Spectrum ← you are here	Three patterns for server-push communication — long polling, server-sent events, and webhooks. Here's how each works and when to reach for each. (150 chars)
09	Sync vs Async Communication: The Architectural Fork	Synchronous services couple tightly. Asynchronous services decouple — but add complexity. Here's how to reason about which your system needs. (147 chars)
10	API Gateways: One Entry Point, Every Cross-Cutting Concern	An API gateway centralises auth, rate limiting, routing, and observability for all your services. Here's what it does, how it works, and when you need one. (158 chars)
11	APIs & Communication: Wrap-Up	A complete recap of all ten API and communication concepts — REST, auth, JWT, WebSockets, webhooks, async patterns, and API gateways — and how they connect. (161 chars)

HTTP/2 multiplexing is SSE's significant advantage over WebSockets for high-client-count systems: multiple SSE streams share a single HTTP/2 connection. A dashboard with ten event streams can use ten SSE connections multiplexed over one TCP connection, while WebSockets would require ten separate TCP connections.

In the URL shortener: SSE is the right choice for the real-time dashboard. The dashboard receives click updates (server-to-client only) — no bidirectional communication needed. One SSE connection per dashboard user, multiplexed over HTTP/2, with automatic reconnection. Significantly simpler to operate than WebSockets for this use case.

---

Webhooks

How it works

Webhooks invert the direction of the HTTP request. Instead of the client polling your server for updates, your server makes an HTTP POST request to the client's endpoint when an event occurs.

Event occurs on your platform:
  [New short link created by user 4821]
                    │
                    ▼
  Your webhook dispatcher:
  POST https://partner-analytics.com/webhooks/shortlinks
  {
    "event": "link.created",
    "data": {
      "id": "x7Kp2",
      "destination": "https://example.com",
      "created_by": "4821",
      "created_at": "2026-01-15T09:30:00Z"
    },
    "webhook_id": "wh_abc123",
    "delivered_at": "2026-01-15T09:30:00.142Z"
  }

  Partner's server processes event and responds:
  HTTP 200 OK

Reliability requirements for webhooks

Webhooks are fire-and-mostly-forget — but "mostly" isn't good enough for production. Several reliability concerns must be addressed:

Delivery guarantees. Networks fail. The recipient's server may be down. Your webhook dispatcher needs a retry strategy: exponential backoff (retry after 1s, 2s, 4s, 8s...) up to a maximum retry count or time window. After exhausting retries, alert the consumer or store the event for manual replay.

Idempotency. Retries mean the same event may be delivered more than once. Consumers must be idempotent — processing the same event twice should produce the same result as processing it once. Webhook payloads should include a unique webhook_id that consumers can use to detect and discard duplicates.

Signature verification. The consumer's endpoint is a public URL receiving HTTP requests. Anyone could send requests to it. Webhook payloads should be signed — your server computes an HMAC of the payload using a shared secret and includes it in a header. Consumers verify the signature before processing.

# Signing (your server):
import hmac, hashlib
signature = hmac.new(
    webhook_secret.encode(),
    payload_bytes,
    hashlib.sha256
).hexdigest()
headers["X-Webhook-Signature"] = f"sha256={signature}"

# Verification (consumer's server):
expected = f"sha256={hmac.new(secret, payload, hashlib.sha256).hexdigest()}"
if not hmac.compare_digest(received_signature, expected):
    return 401  # Reject unverified webhook

Timeout handling. Your webhook dispatcher should set a short timeout (5–10 seconds) for the consumer's response. If the consumer takes too long, treat it as a failure and retry. Consumers should acknowledge receipt immediately (return 200) and process asynchronously — not make the webhook dispatcher wait for processing to complete.

Consumer unavailability. Consumers go offline. Your dispatcher needs a dead-letter queue for events that exhausted retries, and a replay mechanism so consumers can catch up after an outage.

Webhook management for consumers

Any platform offering webhooks needs to provide consumers with:

• Endpoint registration — where to send events
• Event type selection — which event types to subscribe to
• Secret rotation — updating the shared signing secret without downtime
• Delivery logs — visibility into what was sent, when, and whether it succeeded
• Manual replay — triggering redelivery of specific events
• Test delivery — sending a test payload to verify endpoint configuration

In the URL shortener: third-party analytics integrations register webhook endpoints and subscribe to link.created, link.clicked (batched), and link.deleted events. The platform signs every delivery with the consumer's shared secret. Failed deliveries retry with exponential backoff. Consumers receive a management UI showing delivery history and a replay button for failed events.

---

Choosing the right pattern

Does the client need to send messages to the server?
  └─ Yes → WebSockets

Does the client live in a browser or a server?
  └─ Browser, server-to-client push only:
      └─ High frequency (sub-second) or persistent stream → SSE
      └─ Low frequency, simple → Long polling
  └─ Server with its own endpoint:
      └─ Event-driven, fire-and-forget style → Webhooks

Is the client always online?
  └─ No (mobile, offline-capable) → Push notifications or webhooks with queuing

---

The tradeoffs

Long polling vs SSE: SSE is strictly better for server-to-client push in browsers — simpler to implement, automatic reconnection, HTTP/2 multiplexing. Long polling's advantage is universal HTTP compatibility and simpler server-side state management (each request is complete). Use SSE unless compatibility requirements force long polling.

SSE vs webhooks: SSE requires the client to maintain a connection; webhooks don't. For server-to-server event delivery where the consumer can't maintain a long-lived connection or needs asynchronous processing, webhooks are the right pattern. For browser clients that need a persistent stream, SSE.

Webhook reliability complexity: implementing webhooks correctly — retries, idempotency, signature verification, dead-letter queues, replay — is more work than it appears. Using a managed webhook delivery service (Svix, Hookdeck) or a message broker (Kafka, SQS) as the delivery backbone significantly reduces this complexity.

---

The one thing to remember

Long polling, SSE, and webhooks each solve "the server needs to tell the client something" in a different context. SSE is the default for browser clients that only need server-to-client push — it's HTTP-native, auto-reconnecting, and HTTP/2-friendly. Webhooks are the default for server-to-server event delivery — the consumer exposes an endpoint, your platform pushes to it, with retries and signatures. Long polling is the fallback for constrained environments. Know the pattern that fits the context rather than defaulting to WebSockets for everything real-time.

---

← Previous: WebSockets — the auth posts covered how services verify identity; WebSockets cover how clients and servers communicate in real time when HTTP's request/response model isn't enough.

→ Next: Sync vs Async Communication — so far we've covered how to communicate; the next post covers the architectural choice of whether to wait for the response.