Abhiraj K.

CDC Lakehouse Pipeline

• Built an end-to-end CDC pipeline streaming PostgreSQL inserts, updates, and deletes through Debezium and Kafka into Apache Iceberg tables with 1.03s P95 latency. • Integrated MinIO, Hive Metastore, and Trino for queryable lakehouse analytics; containerized a 7-service stack for reproducible deployment.

Databricks Taxi Analytics Pipeline

• Built an incremental Bronze–Silver–Gold pipeline processing 355M+ taxi trip records with schema normalization, data quality filtering, and deduplication. • Designed partition-aware incremental queries that cut data scanned by 89% and execution time by 33%, producing 3 analytics-ready Gold datasets.

Real-Time Crypto Data Engineering Pipeline

• Built a real-time pipeline streaming CoinGecko market events through Kafka into Amazon S3 and Snowflake via Snowpipe. • Developed incremental dbt transformations with MERGE-based upserts across RAW, STAGING, and GOLD layers, orchestrated with Airflow.