Santekno
Projects/Voucher Seller Data Dual-Write to the E-Commerce Platform
ByteDance logoPart of ByteDance · Software Engineer, Promotion Platform Team
Backend / APILive

Voucher Seller Data Dual-Write to the E-Commerce Platform

Reliably dual-wrote voucher seller data to both the legacy marketplace and the e-commerce platform, keeping the two stores consistent with idempotent writes and reconciliation.

2024-mvc-seller-doublewrite · main
# E-Commerce / Data Replication
$ go -X POST /2024-mvc-seller-doublewrite/v1/sync
→ 201 Created
{ "status": "ok" }
publish 2024-mvc-seller-doublewrite.synced
Role
Backend Engineer (Main POC / Owner)
Timeline
2024
Team
4
Domain
E-Commerce / Data Replication
Stack
9 tech
Status
Live
01

Overview

The Voucher Seller Data Dual-Write to the E-Commerce Platform project replicated voucher seller data from a legacy marketplace into the e-commerce platform, keeping both stores consistent. I acted as the main point of contact / owner, starting from technical design and requirements assessment under a tech lead's leadership, and delivered the backend that dual-wrote records reliably at scale.

A backend has no screens, so its "UI" is the operational surface: the dual-write API, the reconciliation/ops flow, and consistency dashboards.

API contract — voucher seller dual-write
Ops view — reconciliation & drift detection
Grafana — dual-write success rate & lag
03

Problem & Solution

Problem

Voucher seller data lived in a legacy marketplace store but also had to be present and consistent in the e-commerce platform store. A naive one-off copy could not guarantee consistency: retries, partial failures, and concurrent updates risked divergence, duplication, or data loss between the two platforms.

Solution

I authored the TRD, the logic architecture, and the database relationship design, then built a dual-write flow: a voucher-seller-service emits change events that a dual-write-service consumes, validates, and writes idempotently to both the legacy marketplace store and the e-commerce platform store. A reconciliation job compares both sides and repairs drift. I monitored the pipeline with Prometheus/Grafana, applied hotfixes to resolve issues promptly, and collaborated with teammates who joined the project.

04

Workflow

Three actors interact with the service; the diagrams are written in English to stay consistent across both languages.

flowchart diagram

create/update voucher

emit change

write

mirror write

reconcile

Voucher Seller

Voucher Seller Service

Dual-Write Service

Legacy Marketplace Store

E-Commerce Platform Store

Reconciliation Job

Use case interaction
sequence diagram
E-Commerce Platform StoreLegacy Marketplace StoreDual-Write ServiceVoucher Seller ServiceE-Commerce Platform StoreLegacy Marketplace StoreDual-Write ServiceVoucher Seller Servicevoucher seller change eventvalidate + build idempotency keywrite recordmirror recordackmark synced / flag drift
Sequence flow
05

Architecture

flowchart diagram

compare

compare

metrics

Voucher Seller Service

Dual-Write Service

Legacy Marketplace Store

E-Commerce Platform Store

Message Queue

Reconciliation Job

Prometheus / Grafana

System architecture
Architecture notes

voucher-seller-service publishes voucher seller changes onto a message queue; dual-write-service consumes them, builds an idempotency key, and writes to both the legacy marketplace store and the e-commerce platform store on Managed RDS. A reconciliation job periodically compares both stores and flags drift for repair. All flows expose metrics to Prometheus/Grafana for consistency and lag monitoring.

state diagram

fields ok

legacy store write

e-commerce store write

both consistent

mismatch detected

reconciled

received

validated

written_legacy

mirrored

synced

drift

Voucher seller record sync lifecycle
06

Features

Seller Data
  • Voucher seller record ingestion
  • Field validation & normalization
Dual-Write
  • Idempotent write to the legacy marketplace store
  • Mirror write to the e-commerce platform store
  • Reconciliation & drift detection
System
  • Database relationship design
  • Monitoring & alerting
  • Hotfix-driven maintenance
07

Tech Stack

language
Go
framework
RPC frameworkHertz
protocol
RPCHTTP
messaging
Message queue
database
Managed RDS (relational database)
observability
PrometheusGrafana
08

Engineering Challenges

01
Keeping two stores consistent

Voucher seller data had to land in both the legacy marketplace store and the e-commerce platform store without divergence. Fix: an idempotent dual-write path with reconciliation that detects and repairs drift, so both platforms stay in sync.

02
No data loss under retries and failures

Partial failures could silently drop or duplicate records. Fix: idempotency keys, ordered change events over a message queue, and a reconciliation job that guarantees eventual consistency with zero data-loss incidents.

03
Owning design under an evolving scope

As the main POC I drove the TRD, logic architecture, and DB relationship design from requirements assessment onward. Fix: a clear write model plus monitoring and hotfix maintenance, iterating with teammates who joined the project.

09

Outcomes

2x
platforms kept in sync
0
data-loss incidents
strong
seller adoption
10

Lessons Learned

Reliable cross-platform replication is less about the write itself and more about idempotency, ordering, and reconciliation. Owning the design end-to-end — from requirements and TRD through DB relationship modeling, monitoring, and hotfix maintenance — is what keeps two platforms consistent with no data-loss incidents.

11

Related Santekno Tutorials

Interested in working together?

Order, payment, or high-traffic event pipelines — let's talk architecture and trade-offs.