All work

01 / Case Study

Creator Subscription & Wallet Platform

Real-time messaging, subscription lifecycles, and a multi-provider wallet and ledger, run across a horizontally scaled NestJS monorepo.

ClientCreator subscription platform
EmployerINK IN CAPS
PeriodDec 2025 — Present
RoleSoftware Development Engineer 2
3,000+concurrent WebSocket connections
$462K/moprocessed across ~7,700 transactions
30K+users served
−75%home-feed load time
−90%home-feed payload size

The problem

The platform sells subscriptions and pays out creator earnings, so two things have to be true at once: chat and notifications need to feel instant across many server instances, and every rupee moving through checkout, renewal, or payout needs to land exactly once — no duplicate charges, no dropped renewals, no earnings that vanish on a retry.

Architecture

The back end is a NestJS 10 monorepo: five deployable apps — core, admin, agency, auth-server, notifications — over seven shared libraries covering auth (JWT per audience plus CASL policy guards), wallet and transaction handling, agency-side transactions, PDF generation, and signed CDN asset URLs. MongoDB (Mongoose 7, snake_case fields, additive-only schema changes — no migration framework) holds the data; Redis and BullMQ carry every background job. Roughly 28 domain modules live under core — posts, stories, channels, groups, feeds, shops, payouts, moderation, complaints, a partner API lane, and payment tracing among them — which is a lot of surface for one service, so the boundaries that matter are the ones drawn around money, media, and real-time rather than around individual features.

Checkout, payment routing, and the wallet ledgerCheckout routes to one of three payment paths, each behind a signature-verified webhook that writes once into a multi-recipient transaction ledger. Ledger writes credit a locked, escrowed balance, which a scheduled cron releases into spendable balance.CheckoutPOST /checkoutProvider Routercard / iDEAL / walletPayment providersCard acquirertokenised cardCentrobilliDEAL / SEPAWalletinternal, no gatewaySigned Webhookidempotent per providerTransaction Ledgerfrom + to[] rowslocked_balanceearnings escrowbalancespendablerelease cronidempotency boundary — replay-safe

Real-time

Sockets are authenticated at the handshake, before a connection can join any room — an unauthenticated socket never reaches a handler. Beyond that boundary sit two stateful handlers, both keeping their state in Redis rather than in process memory, because any instance may serve any user.

  • —Presence tracks multiple concurrent sessions per user — the same account on a phone and a laptop is one presence, not two — across online-public, online-private and live states, with a preference to hide status entirely. Sessions are re-synced and verified on boot so a restarted instance doesn't leave ghosts online.
  • —Live streams hold a viewer set per stream, aggregate reactions through a throttle so a burst of taps doesn't become a burst of emits, and expire on two timers: an idle auto-expire and a longer graceful window for a stream that drops mid-broadcast.
  • —The Redis adapter republishes every emit to sibling instances, so a user connected to one server receives events raised on another. That is what makes the socket tier horizontally scalable instead of sticky.
  • —Notifications branch two ways from the same event: Firebase multicast for device push, and a persisted store backing in-app read/unread counts and acknowledgement.
Socket handshake auth, presence, and cross-instance fan-outA client socket is authenticated with a JWT at the handshake before joining any room. Presence and live-stream handlers keep their state in Redis, and the Redis adapter republishes every emit to sibling API instances so a user connected to one server receives events raised on another. Notifications branch separately to Firebase multicast for push and to a persisted store for in-app read state.Client Socketmobile / webHandshake AuthJWT, pre-joinSocket handlersPresencemulti-session per userLive Streamsviewers · throttled reactionsRedisstate + pub/sub adapterInstance Bsibling socketsInstance Csibling socketsNotification pathFirebasemulticast pushNotification Storeread / unread statefan-outunauthenticated sockets never reach a handler

Async & queues

Every slow or failure-prone operation runs on BullMQ rather than in the request path. The topology is deliberately wide — twelve-plus dedicated queues rather than one general worker — because the queues are separated by failure domain, not by convenience.

  • —Media: asset-optimizer and a separate image/audio lane, so a large video transcode can't sit in front of an avatar resize.
  • —Money: wallet transactions on their own queue, isolated from anything that could back it up.
  • —Messaging: message, response, and unread-message-count queues on a separate Redis connection from the main lane.
  • —Platform: event bridge, scheduled task, channel, and cron; plus security-email and account-email split apart so a marketing backlog can never delay a password-reset mail.
  • —The reason for the split is operational: a stuck media job must not delay a subscription renewal. One shared worker pool makes that failure mode unavoidable; separate queues make it a non-event.

Decisions

Idempotency at the money boundary

Checkout unifies five payment paths — card, two SEPA/iDEAL providers, a legacy gateway, and an internal wallet — behind one endpoint, with each provider's webhook signature-verified independently before it can touch a transaction. Renewals and retries are guarded so a webhook replay or a retried job can never double-charge or double-credit.

Wallet as two buckets, not one

Spendable balance and locked (escrowed) creator earnings are separate fields, moved between by a scheduled release job rather than by application logic reaching into both. That separation is what makes a partial refund, a chargeback, or a wallet-first purchase safe to reason about independently.

Queues own everything slow

Renewal processing, invoice generation, and notification delivery all run through BullMQ rather than inline in the request path, so a slow downstream call (a payment provider, an email service) never holds open an HTTP response.

Stack

Service
NestJS 10TypeScriptMongoDBMongoose 7Monorepo (5 apps, 7 libs)
Real-time
Socket.IORedis adapterRedis Pub/SubPresenceLive streams
Async
BullMQRedis12+ dedicated queuesCron
Auth
JWT per audienceCASL policy guardsSocket handshake authWebhook signature guards
Notifications
Firebase multicastIn-app storeTransactional email
Media
S3sharpffmpegAutomated moderationSigned CDN URLs
Cloud
AWS EC2ALBRoute 53LambdaSESS3