Abimanyu Manohar

Results-driven Data Engineer with 4+ years of IT experience. Experienced in designing, developing, and maintaining scalable ETL/ELT pipelines using PySpark, Python, Databricks and Azure technologies for processing large-scale structured and semi-structured data. Hands-on experience with Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2), Azure SQL Database, and cloud-based data ingestion, storage, and processing solutions.

Proficient in PySpark, Spark SQL, Python, and SQL for data extraction, transformation, validation, cleansing, and optimization. Strong knowledge of Apache Spark, Databricks, Data Lakes, Data Warehousing, Data Modeling, ETL/ELT, Data Ingestion, Data Validation, and Performance Tuning. Experienced in working with Parquet, CSV, JSON, and relational databases including MySQL, Azure SQL Database, Azure Synapse Analytics, and Snowflake.

Familiar with Git, Linux, Jira, Agile/Scrum, CI/CD and version control practices for collaborative development and deployment. Strong analytical and problem-solving skills with experience building reliable, scalable, and high-performance data engineering solutions that support business intelligence and analytics.

Data Engineer at GROWSERVE E-SOLUTIONS PVT LTD (2022-04 – Present)

Project: Electronic Health Records (EHR) Data Integration Platform Technologies: Python, PySpark, SQL, Apache Spark, ETL/ELT, Data Quality, Data Validation, Dimensional Modeling, Star Schema, Fact & Dimension Tables, Incremental Loading, CSV, Linux/Ubuntu

• Developed scalable ETL/ELT pipelines to integrate Patients, Doctors, Departments, Appointments, Admissions, Billing, and Pharmacy data from multiple sources.
• Used Python, PySpark, and SQL for data ingestion, transformation, cleansing, validation, and large-scale processing.
• Implemented data quality checks to identify and handle duplicates, null values, invalid records, inconsistent formats, and referential integrity issues.
• Designed fact and dimension tables using dimensional modeling to support healthcare analytics and reporting.
• Developed SQL transformations using joins, CTEs, subqueries, aggregations, CASE statements, and window functions.
• Implemented incremental loading and source-to-target validation to improve pipeline efficiency and data accuracy.
• Performed data reconciliation and validation using record counts, key-field checks, and business-rule validations between source and target datasets.
• Troubleshot pipeline failures, schema mismatches, transformation issues, and data-quality exceptions while documenting data mappings and transformation logic.

Data Engineer at GROWSERVE E-SOLUTIONS PVT LTD (2022-04 – Present)

Project: Enterprise CDC & Data Integration Platform Technologies: Python, PySpark, SQL, Apache Spark, ETL/ELT, Data Quality, Data Validation, Dimensional Modeling, Star Schema, Fact & Dimension Tables, Incremental Loading, CSV, Linux/Ubuntu

• Developed scalable ETL/ELT pipelines for integrating data from multiple enterprise source systems.
• Implemented CDC-based incremental pipelines to capture INSERT, UPDATE, and DELETE operations and propagate changes to downstream systems.
• Built data transformation workflows using SQL, Python, and PySpark for large-volume datasets.
• Implemented data cleansing, deduplication, standardization, and data quality validation processes.
• Developed incremental loading strategies to minimize full-load processing and improve pipeline performance.
• Designed and maintained fact and dimension tables for analytical workloads and reporting.
• Performed source-to-target reconciliation using record counts, key-field validation, and business-rule checks.
• Implemented error handling, logging, and monitoring to identify and troubleshoot pipeline failures and data inconsistencies.
• Optimized SQL queries and Spark transformations to improve processing efficiency and scalability.
• Created and maintained documentation for data flows, source-to-target mappings, transformation rules, and pipeline dependencies.

B.E in Mechatronics – Coimbatore Institute of Engineering and Technology, Coimbatore (2013 – 2017)