REST vs Webhooks vs Polling: Choosing an Integration Pattern

Integrations fail when the pattern does not match the event. Teams poll a payment API every five seconds “to be real-time,” or they wait for a webhook that the vendor sends once and never retries. Palmate’s API integration work starts by naming the trigger: user action, vendor event, or a clock.

For the operational checklist, see API integration checklist for businesses. For the failure modes, see common mistakes when integrating third-party APIs.

Three patterns, three jobs

On-demand REST is a request you make because a person or a job needs an answer now: create a shipment, fetch a rate, validate a GSTIN. Latency is the user’s. Errors are visible. This is the right default for actions, not for “keep two databases identical.”

Polling is a scheduled pull: ask the vendor what changed since a cursor. It is honest when the vendor has no webhooks, webhooks are unreliable, or you must reconcile. It costs rate limit budget and lag. It is not a design smell. It is how most accounting and marketplace APIs actually work.

Webhooks are the vendor calling you when something happened. They are efficient when the vendor implements signatures, retries, and a delivery log. They are a public write endpoint on your side. Treat them as untrusted input.

Most production systems use all three: REST for commands, webhooks for hints, polling for catch-up.

Match the pattern to the business event

Ask what “late” costs.

• Payment captured — late is refunds and angry finance. Prefer webhooks plus a reconcile poll. Do not rely on a browser redirect alone.
• Catalogue price change — late is a wrong quote. A nightly poll may be enough; a webhook is nicer.
• Stock on a marketplace — late is oversell. You need events and a reservation model; see how e-commerce inventory synchronisation works. Copying a number on a timer will not save you.
• CRM note — late is a salesperson looking silly. Polling every hour is often fine.

If you cannot name the cost of lag, you will over-engineer webhooks for a report nobody reads until Monday.

Webhooks are not free real-time

You must:

• Verify signatures. Unsigned webhooks are an open API.
• Acknowledge fast; do slow work in a queue.
• Store event ids so retries do not double-apply.
• Have a replay path when your mapping was wrong.

If the vendor cannot show a delivery log or retry policy, assume you need polling as the source of truth. The webhook is a cache invalidation, not a ledger.

Pretty-print sample payloads with the JSON formatter before you write handlers. “Almost JSON” wastes a week.

Security hardening for incoming webhooks

Exposing an unauthenticated webhook endpoint to the public internet turns your server into an arbitrary write target. A resilient receiver implements three cryptographic layers:

1. HMAC Signature Verification with Timing-Safe Equality: Always compute the HMAC-SHA256 of the raw incoming request body against your shared secret. Never use standard string equality (=== or strcmp), which is vulnerable to side-channel timing attacks. Use constant-time comparison (e.g., crypto.timingSafeEqual in Node.js).
2. Timestamp Verification & Replay Protection: Attackers who capture a legitimate webhook payload can replay it repeatedly. Reliable vendors send a timestamp header (e.g., Stripe-Signature: t=1614...). Verify that the timestamp is within 5 minutes of your server's current UTC clock. If it drifts further, reject the request immediately.
3. Raw Body Integrity: Verification must run on the exact raw bytes before JSON parsing. Middleware that modifies whitespace or parses JSON into JavaScript objects prior to signature verification will cause intermittent signature validation failures that are brutal to debug.

import crypto from "crypto";

export function verifyWebhookSignature(rawBody: Buffer, signature: string, secret: string): boolean {
  const expected = crypto.createHmac("sha256", secret).update(rawBody).digest("hex");
  const signatureBuffer = Buffer.from(signature, "utf8");
  const expectedBuffer = Buffer.from(expected, "utf8");
  if (signatureBuffer.length !== expectedBuffer.length) return false;
  return crypto.timingSafeEqual(signatureBuffer, expectedBuffer);
}

Dead-letter queues and replay pipelines

Webhooks arrive out of order and during database maintenance windows. If your receiver does the business logic inside the HTTP handler, a database timeout returns a 500 status code. The vendor retries with exponential backoff — or stops retrying altogether after three attempts.

The standard architectural pattern is a thin ingest buffer:

1. Acknowledge in under 200ms: Validate the HMAC signature, write the raw payload to an append-only message queue (such as RabbitMQ, AWS SQS, or Redis Stream), and return a 200 OK or 202 Accepted immediately.
2. Idempotency table check: The background consumer checks event_id in an idempotency table with a unique constraint. If already processed, it acknowledges and discards safely.
3. Dead-Letter Queue (DLQ): If business processing fails after 3 retries (due to unexpected JSON formatting or downstream API failures), route the event to a dead-letter queue.
4. Replay UI or CLI: Build a command that lets an engineer re-dispatch failed events from the DLQ after deploying a bug fix, without asking the third-party vendor to resend their webhook history.

Polling needs cursors, not page 1

Offset pagination that you restart from zero every night will miss updates and burn limits. Prefer updated_since or opaque cursors. Test a complete crawl, not a happy path of forty rows.

Backoff on 429. Honour Retry-After. A tight loop during a sale is how you get banned.

REST on a user action still needs a contract

Idempotent POSTs, timeout budgets, and error bodies that an operator can read belong here too. API error handling operations can use is the companion.

Count flows with the API project estimator before you promise “we will just webhook Shopify.” Shopify is not one flow.

Operational cost and network budget

How API integration can automate business operations is the why. Pattern choice is the how. If the design doc says “real-time” with no lag budget and no reconcile job, it is not a design. It is a slogan.

Authoritative References & Standards

To cross-reference the engineering patterns and regulatory considerations described in this guide, consult the following authoritative industry documentation and RFC standards:

• IETF RFC 7231: Hypertext Transfer Protocol (HTTP/1.1) Semantics and Content — The foundational standard for HTTP request methods, idempotency, and status codes.
• OpenAPI Specification (OAS) — The standard machine-readable interface definition for RESTful APIs.
• IETF RFC 7519: JSON Web Token (JWT) — Industry specification for securely representing claims between parties.