Case study · New Zealand restaurant platform

Sales ingest for a 180-store fast-food franchise group in New Zealand

Every night each store's point-of-sale system sends its sales, and by morning every one of them has to be in the ledger. I moved that ingest off a synchronous request path onto S3, SQS FIFO and autoscaled Fargate workers, then load-tested it until it held at 180 stores.

ClientA fast-food franchise group in New Zealand, rolling out to 180 stores
EmployerPrevious company, as a contract software engineer
ProductAn AWS and TypeScript platform replacing a legacy low-code back office for the franchises
How I built itWith Claude Code agents, working from a spec, a plan and tests. I reviewed every change before it merged.

Problem

Store sales arrive as XML bundles. The old path parsed each bundle inside the HTTP request, in a single 930-line database transaction. It managed 3.75 bundles a minute, which was too slow for a 180-store rollout, and long requests were timing out at the load balancer. Two pushes for neighbouring days from the same store could also race each other, which left some sales days silently underived.

Constraints

Architecture

  1. Store point-of-saleNightly XML sales bundle, sent through a relay
  2. Accept endpointStore raw bytes in S3, write a pending row, reply fast
  3. SQS FIFO queueOne message group per store, so each store's days stay in order. Dead-letter queue behind it.
  4. ECS Fargate workersAutoscaled. Claim the pending row, parse the bundle, derive ledger events.
  5. Postgres event ledgerMulti-tenant, row-level security
  6. Scheduled sweepEventBridge job that finds sales days which never completed
    Alarms and recoveryCloudWatch metrics and alarms, plus a support console to replay a failed ingest
  • Self-healing check
  • Operator path
Keying the FIFO message group by store removed the neighbouring-day race. The request path only stores and enqueues.

What I did

I re-architected the accept path so it does very little. It writes the raw bundle to S3, records a pending row, and sends one SQS FIFO message keyed by store. A pool of autoscaled Fargate workers picks up the messages, claims the row, parses the XML and derives the ledger events. A scheduled EventBridge sweep catches any sales day that didn't complete, and support staff get a console to replay a failed ingest without engineering help.

The first 180-store load test failed. p99 accept latency was 13.3s. I worked through the CloudWatch metrics and found three root causes. S3 writes were happening inside a locked database transaction. A version check parsed the whole XML file just to read one attribute. And the autoscaler was scaling in workers that were still busy. I moved the S3 writes out of the lock, replaced the full parse with a root-tag scan (507ms to 10ms at p50), and stopped busy workers from being scaled in. Together that cut p99 accept latency 8x.

Afterwards I extracted the reusable CDK constructs and the async-ingest runtime into shared npm packages and moved consumers across 4 repositories onto them. Synth-diffing 14 stack and environment pairs showed zero infrastructure drift.

Results

Measured in a 180-store, 7-day k6 load test of 1,260 bundles at 5 MiB each.

13.3s to 1.63s
server-side p99 accept latency (8x)
3.75 to 13-15
bundles per minute
0
dead-lettered messages
0
gateway timeouts (504s)
0 of 1,080
site-days silently dropped
507ms to 10ms
version check at p50 after the parse fix

On real traffic, accept latency went from 4.5-7s to 0.7-1.1s.

Stack

Timeline

  1. Designed the queue-and-worker ingest and the per-tenant feature flag.
  2. Built the accept path, FIFO queue, Fargate workers, sweep and alarms.
  3. First 180-store load test: p99 accept latency of 13.3s, a fail.
  4. Traced the three root causes and fixed them.
  5. Rerun of the 180-store, 7-day test: p99 of 1.63s, 0 dead letters, 0 dropped site-days.
  6. Extracted the pattern into shared packages used across 4 repositories.

The client is described by country and industry instead of by name.

Back to case studies