Pankaj Ojha
Software Developer
Hire their services
Request a quote with no obligation
About
Data Engineer with solid experience designing and optimizing scalable data pipelines, ETL workflows, and cloud-based architectures across Azure and Google Cloud. Skilled in Apache Spark (Scala/PySpark), SQL, data modeling, and distributed computing, with hands-on work using Databricks, Azure Data Factory, and Airflow to orchestrate and automate complex workflows. Experienced with data lakes, Delta Lake, and cloud-native databases to support reliable analytics and reporting.
Additionally, I have practical exposure to Generative AI, including RAG, embeddings, and agentic workflows, applying them to automate analysis and enhance data understanding within modern data platforms.
Experience
- Designed and maintained scalable cloud ETL pipelines using Azure Data Factory, Databricks, and GCP BigQuery, processing 50M+ records across sales and inventory systems.
- Built Spark-based incremental pipelines (SCD Type 2) orchestrated with Airflow, reducing data latency and preserving full historical data
- Developed PySpark data cleansing, aggregation, and testing frameworks, improving data quality, pipeline reliability, and analytical performance
Education
B.Tech in Electronics and Communication Engineering.