Santekno
Projects/Merchant Voucher Aggregator — Realtime Filtered Voucher Feed
Tokopedia logoPart of Tokopedia · Software Engineer, Shop Team
Backend / APILive

Merchant Voucher Aggregator — Realtime Filtered Voucher Feed

Backend for a realtime voucher feed powered by flexible filters (thematic, shop tier, category, voucher type, target audience), with a Chip Filter addition shipped in under a month by enhancing existing entry points and a background updater flow that kept the front panel fast at roughly 24k RPS.

2022-mvc-agregator-filtered · main
# E-Commerce / Promotion & Discounts
$ go -X POST /2022-mvc-agregator-filtered/v1/sync
→ 201 Created
{ "status": "ok" }
publish 2022-mvc-agregator-filtered.synced
Role
Backend Engineer
Timeline
2022
Team
4
Domain
E-Commerce / Promotion & Discounts
Stack
8 tech
Status
Live
01

Overview

The Merchant Voucher Aggregator powers a realtime voucher feed on the merchant voucher platform, driven by flexible filters — thematic, shop tier, category, voucher type, and target audience. As backend engineer leading the Chip Filter addition, I extended the aggregator so a new front-panel filter could plug into the same read path used by the discovery-page and home-page teams, and shipped it end-to-end in under a month for a November 2022 launch.

A backend has no screens, so its "UI" is the operational surface: the aggregated voucher read API, the chip filter contract with the discovery-page and home-page teams, and the aggregator latency dashboards.

API contract — filtered voucher read (chip filter)
Ops view — filter dimensions & background refresh
Grafana — voucher feed RPS, latency & success rate
03

Problem & Solution

Problem

The voucher feed serves the public read path at roughly 24k RPS and had to gain a new Chip Filter dimension without adding a parallel pipeline, without regressing latency on the front panel, and without breaking the existing five-filter contract that the discovery-page and home-page teams already depended on.

Solution

I extended the existing aggregator entry points instead of introducing a new flow, so the Chip Filter reused the same request contract, filter resolution, and cache surface as the other filter dimensions. To keep the front panel fast, I designed and implemented a background updater flow that periodically refreshes and warms the filtered feed cache — so live reads hit a precomputed result rather than recomputing on the request path. I aligned with the discovery-page and home-page teams on a single filter contract and staged rollout, and the feature launched in November 2022 in under a month of build time.

04

Workflow

Three actors interact with the service; the diagrams are written in English to stay consistent across both languages.

flowchart diagram

opens voucher feed

apply chip filter

read cached feed

refresh feed

rpc

metrics

Buyer

Front Panel

Voucher Aggregator Service

Redis Feed Cache

Background Updater

Promotion Service

Prometheus / Grafana

Use case interaction
sequence diagram
Background UpdaterRedis Feed CacheVoucher Aggregator ServiceFront PanelBuyerBackground UpdaterRedis Feed CacheVoucher Aggregator ServiceFront PanelBuyeropen voucher feed (chip filter selected)GetPublicVoucher(filters)read filtered feedcached voucher listvoucher list (low latency)periodic refresh triggerrecompute & warm cache
Sequence flow — filtered read with background refresh
05

Architecture

flowchart diagram

rpc

rpc

metrics

Front Panel · Discovery Page / Home Page

Voucher Aggregator Service · Go

Redis Feed Cache

Promotion Service

Catalog Service

Background Updater

Prometheus / Grafana

System architecture
Architecture notes

The voucher aggregator service (Go) owns the realtime feed and exposes a filtered read API consumed by the discovery-page and home-page front panels. Reads are served from a Redis feed cache, keyed by the active filter combination (thematic, shop tier, category, voucher type, target audience, plus the new chip filter). A background updater recomputes filtered feeds on a schedule and warms the cache, so the request path stays cache-first. Downstream RPCs to promotion and catalog services are only touched during refresh cycles or on cache miss. Prometheus and Grafana track RPS, latency, and success rate on the public voucher read path.

state diagram

filters resolved

background updater refreshed

served on front panel

TTL elapsed

next refresh cycle

configured

indexed

warmed

served

stale

Voucher feed entry lifecycle
06

Features

Realtime Feed
  • Realtime voucher aggregation across five filter dimensions
  • Thematic, shop tier, category, voucher type, target audience filters
  • Chip Filter addition for the discovery-page and home-page front panels
Performance & Latency
  • Background updater flow to optimize front-panel latency
  • Cache-first read path serving roughly 24k RPS
  • Sub-second reads with a 99% success rate on the public voucher path
Delivery
  • Enhanced existing entry points to avoid duplicating flows
  • Shipped end-to-end in under one month
  • Coordinated with discovery-page and home-page teams
07

Tech Stack

language
Go
framework
RPC framework
protocol
gRPCRPCHTTP
cache
Redis
observability
PrometheusGrafana
08

Engineering Challenges

01
Adding a new filter without a new pipeline

The Chip Filter had to plug into an already-live voucher feed without spawning a parallel read path or destabilizing throughput at scale. Fix: enhanced the existing aggregator entry points so the new filter reused the same contract and cache surface, letting the addition ship end-to-end in under a month.

02
Keeping the front panel fast under load

The discovery-page and home-page front panels needed low-latency reads on top of a public voucher path serving roughly 24k RPS. Fix: a background updater flow that periodically recomputes and warms the filtered feed cache, so live requests hit a ready result and stay well below the previous averages.

03
Aligning two front-end teams on one backend contract

Two front-panel teams — discovery-page and home-page — needed the same filter behavior but had slightly different rollout timelines. Fix: a single filter contract with clear filter dimensions (thematic, shop tier, category, voucher type, target audience), staged rollout, and shared dashboards so both surfaces launched cleanly in November 2022.

09

Outcomes

<1 mo
shipped end-to-end
24k RPS
read throughput
5 filters
flexible filter dimensions
10

Lessons Learned

Enhancing existing entry points instead of forking a new flow was the single biggest reason the Chip Filter shipped end-to-end in under a month — the same read contract, the same cache surface, and the same observability all came for free. And moving refresh work off the request path into a background updater is a simple, durable way to keep a public voucher feed serving roughly 24k RPS at a 99% success rate without paying for it on every read.

11

Related Santekno Tutorials

Interested in working together?

Order, payment, or high-traffic event pipelines — let's talk architecture and trade-offs.