Skip to content

Case study · Logistics · Data analytics

How NimbusPost Partnered with Shellkode for Near Real-Time Logistics Analytics

NimbusPost · Logistics technology

Analytics was capped by data that sat in silos and arrived late. ShellKode built NimbusPost a platform that streams it, transforms it and surfaces it within minutes of an event.

  • 1.65 TB

    Data unified from MySQL and MongoDB

  • 50 GB

    New data a month captured by CDC

  • ~300

    KPIs across 20 dashboards

  • 263

    Source tables ingested (214 MySQL, 49 MongoDB)

The opportunity

Millions of daily shipments, but 1.65 TB of data locked in MySQL, MongoDB and CRM silos.

NimbusPost is one of India's fastest-growing logistics technology platforms. It connects online sellers with multiple courier partners for nationwide delivery, and it handles millions of shipment transactions a day, from pickup scheduling and courier assignment to tracking updates, delivery attempts and returns.

At that scale, data is not a byproduct of operations. It is the foundation on which every business decision is made. But NimbusPost's data sat in MySQL (around 900 GB), MongoDB (around 750 GB) and roughly 10 CRM APIs with no central repository, no near real-time visibility and no analytics platform to build on.

More reports were never going to fix that. A different architecture would. NimbusPost needed a complete data analytics platform built from scratch, and ShellKode built it.

Data in silos

About 1.65 TB of data was spread across MySQL, MongoDB and CRM APIs, with no centralized repository for unified analytics.

No real-time visibility

The business lacked near real-time insight into key operational metrics and shipment performance.

No analytics platform

A complete data analytics infrastructure had to be built from the ground up.

Decisions on stale insights

Order-processing and customer-retention reporting ran on delayed data, so teams reacted to events after the fact.

The solution

A platform that streams, transforms and surfaces continuously.

ShellKode designed and implemented a centralized near real-time data analytics platform on AWS, integrating data from MySQL and MongoDB and making it available for decision-making within minutes of an event occurring. The system does not collect and report. It streams, transforms and surfaces continuously.

Capture changes continuously

Data from the four source databases streams into one central platform as it changes, instead of waiting for batch loads.

Process in near real-time

Data is transformed as it arrives, so it is ready for decisions within minutes of an event.

Organize in one governed data lake

Processed data lands in a single, structured and governed layer built to grow with shipment volumes.

Analyze without waiting on engineering

Business teams query the data and watch near real-time dashboards on their own.

Monitor every layer

Pipeline reliability is tracked, with alerting and performance visibility across every layer.

The impact

Now the business sees what is happening within minutes.

Data from four source databases flows into a centralized Apache Iceberg data lake. Business teams query it and watch dashboards on their own, and every layer of the pipeline is monitored.

  • Near real-time visibility

    Change data capture streams MySQL and MongoDB changes continuously instead of in batch loads, cutting latency from days to minutes, so 20 order-processing and customer-retention dashboards read from one governed source.

  • 70% faster decision-making

    Business teams act on current data rather than stale batch reports.

  • A scalable data lake

    Iceberg-based architecture handles growing data volumes without performance degradation, with built-in schema evolution.

  • Self-service analytics

    Business teams query and visualize data independently, reducing dependency on engineering.

  • Improved data reliability

    Consistent, monitored insights are available across all business functions.

  • Full-stack observability

    Grafana, Prometheus, Kafka UI and EMR YARN Logs give end-to-end monitoring across every layer.

The stack

What it runs on.

Sources
MySQL, MongoDB, CRM APIs
Capture and processing
Change data capture, Amazon EMR
Storage
Apache Iceberg data lake
Observability
Grafana, Prometheus, Kafka UI, EMR YARN logs

Talk to us

Bring one data problem. Leave with a platform design that solves it.

30 minutes on one analytics workflow, whether that's ingestion, processing or dashboards, mapped to the architecture behind it. No slides.