Pricing Data Lake & Pricing API (2022.6-): provide medical prices data with a cost-effective and fast querying API.
• Responsible for data ingestion, data validation, data pipeline automation and data testing.
• Led to design an efficient parser to solve a critical problem of large file ingestion. Ingesting files without out-of-memory issue (size
100GB -1TB/per file, peek memory used 50MB). Enhanced ingestion speed 192x compared to previous design.
• Led to design an automation data pipeline. Monthly ingesting 17 customers’ data and generating 30+TB parquet data. Reduce 30%
AWS cost of idling tasks and 95% human involvement.
Shop Book and Pay (2019-2022.6): serve hospitals to build shop sites for searching and buying medical services. • Implement a pricing data ingestion system with .NET Core, Azure Functions, Blob Storage, and Service Bus. • Used Adobe Experience Manager (AEM) and ReactJS to host 70 shop sites.