Voucher Seller Data Dual-Write to the E-Commerce Platform
Reliably dual-wrote voucher seller data to both the legacy marketplace and the e-commerce platform, keeping the two stores consistent with idempotent writes and reconciliation.
- Role
- Backend Engineer (Main POC / Owner)
- Timeline
- 2024
- Team
- 4
- Domain
- E-Commerce / Data Replication
- Stack
- 9 tech
- Status
- Live
Overview
The Voucher Seller Data Dual-Write to the E-Commerce Platform project replicated voucher seller data from a legacy marketplace into the e-commerce platform, keeping both stores consistent. I acted as the main point of contact / owner, starting from technical design and requirements assessment under a tech lead's leadership, and delivered the backend that dual-wrote records reliably at scale.
Gallery
A backend has no screens, so its "UI" is the operational surface: the dual-write API, the reconciliation/ops flow, and consistency dashboards.
Problem & Solution
Voucher seller data lived in a legacy marketplace store but also had to be present and consistent in the e-commerce platform store. A naive one-off copy could not guarantee consistency: retries, partial failures, and concurrent updates risked divergence, duplication, or data loss between the two platforms.
I authored the TRD, the logic architecture, and the database relationship design, then built a dual-write flow: a voucher-seller-service emits change events that a dual-write-service consumes, validates, and writes idempotently to both the legacy marketplace store and the e-commerce platform store. A reconciliation job compares both sides and repairs drift. I monitored the pipeline with Prometheus/Grafana, applied hotfixes to resolve issues promptly, and collaborated with teammates who joined the project.
Workflow
Three actors interact with the service; the diagrams are written in English to stay consistent across both languages.
Architecture
voucher-seller-service publishes voucher seller changes onto a message queue; dual-write-service consumes them, builds an idempotency key, and writes to both the legacy marketplace store and the e-commerce platform store on Managed RDS. A reconciliation job periodically compares both stores and flags drift for repair. All flows expose metrics to Prometheus/Grafana for consistency and lag monitoring.
Features
- Voucher seller record ingestion
- Field validation & normalization
- Idempotent write to the legacy marketplace store
- Mirror write to the e-commerce platform store
- Reconciliation & drift detection
- Database relationship design
- Monitoring & alerting
- Hotfix-driven maintenance
Tech Stack
Engineering Challenges
Voucher seller data had to land in both the legacy marketplace store and the e-commerce platform store without divergence. Fix: an idempotent dual-write path with reconciliation that detects and repairs drift, so both platforms stay in sync.
Partial failures could silently drop or duplicate records. Fix: idempotency keys, ordered change events over a message queue, and a reconciliation job that guarantees eventual consistency with zero data-loss incidents.
As the main POC I drove the TRD, logic architecture, and DB relationship design from requirements assessment onward. Fix: a clear write model plus monitoring and hotfix maintenance, iterating with teammates who joined the project.
Outcomes
Lessons Learned
Reliable cross-platform replication is less about the write itself and more about idempotency, ordering, and reconciliation. Owning the design end-to-end — from requirements and TRD through DB relationship modeling, monitoring, and hotfix maintenance — is what keeps two platforms consistent with no data-loss incidents.
Related Santekno Tutorials
Interested in working together?
Order, payment, or high-traffic event pipelines — let's talk architecture and trade-offs.