8 min readCategory: Data Engineering
XGBoost vs LightGBM: Benchmarking Tabular Data Processing Speed & Accuracy
Histogram-based decision trees vs depth-wise growth: why LightGBM trains 7x faster on 10M row datasets with equal AUC-ROC scores.
Data Engineering
Field notes on Apache Iceberg vs Delta, Debezium CDC, Snowflake clustering, Parquet dictionary encoding, Flink, and dbt.
7 publications, newest first
8 min readCategory: Data Engineering
MLOps Model Registry Patterns: Tracking Artifacts with MLflow & S3
Automating model lineage, hyperparameter logging, and staging-to-production promotion gates in CI/CD pipelines.
8 min readCategory: Data Engineering
Data Drift Detection in Production ML: Population Stability Index (PSI) & Monitoring
Catching covariate shift and concept drift before model accuracy degrades using PSI metrics and Evidently AI dashboards.
10 min readCategory: Data Engineering
Building CDC Ingestion with Apache Debezium, Kafka, and StarRocks
Streaming PostgreSQL WAL changes into StarRocks real-time analytics engine with sub-second end-to-end latency.
10 min readCategory: Data Engineering
Apache Iceberg vs Delta Lake: Comparing Table Formats for Cloud Data Lakes
Comparing hidden partitioning, ACID transactions, time-travel queries, and multi-engine interoperability across Snowflake and Spark.
8 min readCategory: Data Engineering
Parquet File Format Deep Dive: Row Groups, Dictionary Encoding, and Snappy Compression
Why Parquet files achieve 80%+ compression ratios over CSV: inspecting columnar binary structure, footer metadata, and dictionary encoding.
9 min readCategory: Data Engineering
Real-Time Stream Processing with Apache Flink & Kafka: Event Time vs Processing Time
Handling out-of-order event streams, tumbling vs sliding windows, and watermarking strategies for sub-second streaming analytics.