Skip to content
CodeNest

~/work/billing-payment-grade-data-integrity

A billing event pipeline that never loses or double-charges

Core services published billing events to the broker in-process, coupling every write to broker availability and risking lost or duplicated events. We re-architected the pipeline around change-data capture and end-to-end idempotency.

Industry
Usage-based billing platform
Role
Architecture and implementation
Services
Architecture · Backend development · Distributed systems
Abstract illustration of ledger rows with a rising line chart

Challenge

Core services published billing and usage events to the message broker through in-process calls. That coupled the write path to broker availability and, on failure, risked both lost and duplicated events.

When events meter money and usage, neither is acceptable.

Approach

  • Change-data capture instead of in-process publishing. Re-architected the flow to DynamoDB Streams → a Lambda publisher → the event broker. Services no longer talk to the broker directly, and every committed state change is published downstream.
  • Idempotency end to end. Enforced idempotency with Redis SET NX and added consumer-side deduplication to absorb the duplicates that at-least-once delivery produces.
  • A hardened broker client layer. Migrated to a modern Kafka client, fixed asynchronous sends that were stalling consumer lag, corrected commit semantics, and resolved IAM / authorisation issues around idempotent writes.
  • No more race conditions on deletes. Eliminated a quota-deletion race with a soft-delete + TTL + secondary-index design.

Outcome

Publishing is now driven by committed data rather than by in-process calls, so services deploy and run independently of the broker. Duplicates from at-least-once delivery are absorbed on both the producer and consumer side, and the quota-deletion race is designed out rather than worked around.

Want to go deeper on the idempotency side? We wrote about it in Idempotency in payment systems.

Selected work

All case studies →
Abstract illustration of radio signal rings connecting to distributed device nodes

Telecommunications & IoT

Scaling a real-time telecom charging & session platform

A mobile-core platform had to sustain millions of concurrent connectivity sessions, but under load the charging path became the throughput ceiling and nodes ran out of memory. We traced both to root cause, re-tuned the I/O-bound charging path and delivered a prioritised scaling strategy.

  • Go
  • NATS
  • Kafka
  • Redis / ElastiCache
Abstract illustration of a grid of image, video and document tiles, one being rendered

Media & content platforms

A Digital Asset Management platform, built end to end

A product that needed to store, organise and serve large volumes of media — with fast on-demand rendering and flexible metadata search, without re-processing everything up front. Built end to end, from API to deployment.

  • Go
  • Gin
  • PostgreSQL
  • JSONB

~/next-step

Have something that needs to ship — and keep running?

Tell us what you are building. Within one business day you get a reply from a senior engineer, not a sales rep — with honest questions, a rough approach, and whether we are the right fit.

Book a discovery call[email protected]

// NDA available before the first call.