Prathyush Paruchuri

Azure Data Engineer with 5+ years of dedicated experience architecting and delivering scalable data pipelines, Lakehouse solutions, and ETL/ELT frameworks on Microsoft Azure, backed by 9 years of overall professional experience across technology and infrastructure domains. Proficient in Azure Data Factory (ADF) for orchestrating complex ingestion workflows across on-premises, cloud, and third-party systems using parameterized and metadata-driven frameworks. Hands-on expertise with Azure Databricks (PySpark, Spark SQL, Python) for large-scale data transformations, aggregations, schema evolution, and advanced analytics.

Strong experience implementing Medallion Architecture (Bronze, Silver, Gold) on ADLS Gen2 and Delta Lake — enabling ACID transactions, time travel, and analytics-ready data layers. Skilled in data modelling (Star/Snowflake Schemas, SCD Type 2) and delivering curated datasets via Azure Synapse Analytics and Azure SQL Database. Implemented enterprise-grade security using Azure Key Vault, Unity Catalog, RBAC, and data masking for compliance across healthcare, retail, insurance, and manufacturing domains.

Hands-on with CI/CD using Azure DevOps and GitHub Actions; experienced in Agile/Scrum delivery with cross-functional stakeholder collaboration. Proven track record of optimizing Spark workloads through partitioning, caching, and broadcast joins — achieving measurable improvements in pipeline performance and cost efficiency.

Data Engineer for EPAM Systems (2021-07 – Present)

• Architected an end-to-end cloud-native data platform on Azure to process high-volume insurance claims, policy, and underwriting datasets from heterogeneous sources (MySQL, CSV, JSON, REST APIs).
• Designed and implemented metadata-driven ADF pipelines for full and incremental ingestion using Watermark and CDC strategies, reducing data latency significantly.
• Built scalable PySpark transformation frameworks in Azure Databricks to cleanse, standardize, and enrich TB-scale structured and semi-structured datasets.
• Implemented Medallion Architecture (Bronze, Silver, Gold) on ADLS Gen2 using Delta Lake — enabling ACID transactions, schema evolution, time travel, and auditability.
• Applied Spark performance optimizations (partition tuning, broadcast joins, caching, Z-ordering) to reduce job execution time and pipeline costs.
• Delivered curated Gold-layer datasets to Azure Synapse Analytics for actuarial KPI dashboards and executive reporting.
• Enforced enterprise-grade security via Azure Key Vault, Unity Catalog, RBAC, and column-level data masking for PII and sensitive policy data.
• Automated CI/CD deployments using Azure DevOps across dev, QA, and production environments; implemented proactive monitoring with Azure Monitor and Log Analytics.

Project 4: Healthcare Data Lakehouse — Predictive Analytics Platform

• Designed and orchestrated ADF pipelines to ingest patient, clinical, and EHR data from OLTP/OLAP databases, SFTP, and REST APIs into ADLS Gen2.
• Developed PySpark pipelines in Azure Databricks for deduplication, cleansing, and standardization of large-scale patient datasets, applying SCD Type 2 for accurate historical tracking.
• Built Bronze, Silver, and Gold Delta Lake layers with schema evolution and audit trails, ensuring data consistency and regulatory traceability.
• Created analytics-ready datasets in Azure Synapse Analytics to power clinical KPI dashboards and support predictive model consumption by data science teams.
• Implemented RBAC and data masking on sensitive PHI fields; managed pipeline health monitoring and alerting via Azure Monitor for SLA-compliant production support.

Project 3: Manufacturing Workforce & Operations Management System

• Designed ADF ingestion pipelines to consolidate workforce, operational, and IoT sensor data from on-premises and cloud source systems into ADLS Gen2.
• Developed PySpark-based data transformations in Azure Databricks to clean, standardize, and aggregate operations datasets; implemented Bronze-Silver-Gold Medallion layers with Delta Lake.
• Created analytics-ready Star Schema models in Azure Synapse Analytics to support workforce productivity KPIs and operational performance reports.
• Applied incremental processing, data quality checks, and schema validation to ensure reliability and completeness of production data outputs.
• Published curated datasets to Power BI enabling real-time operational dashboards for plant managers and business stakeholders.

Project 2: Retail Inventory & Supply Chain Analytics Platform

• Built ADF ingestion pipelines to consolidate POS, ERP, and inventory data from multiple retail source systems into ADLS Gen2 using watermark-based incremental loading.
• Developed PySpark transformation frameworks in Azure Databricks for data standardization, deduplication, and aggregation; designed Bronze, Silver, and Gold Delta Lake layers.
• Implemented data quality and reconciliation checks to validate inventory accuracy and ensure consistency across reporting layers.
• Created dimensional models (Star Schema) in Azure Synapse Analytics enabling store-level, regional, and product performance analysis for business users.
• Automated pipeline scheduling and dependency management in ADF; optimized Spark workloads resulting in improvement of pipeline execution time.

Project 1: Trade & Sales Data Warehouse — SQL Development & ETL

• Developed and optimized complex T-SQL stored procedures, views, and indexes to support data transformation and normalization for sales and inventory reporting.
• Built ADF pipelines to ingest structured data from SQL databases and flat files into Azure Blob Storage and Azure SQL Database with full and delta load strategies.
• Implemented data quality validations and reconciliation checks within SQL layers to ensure downstream reporting accuracy.
• Managed database scripts, version control via Git, and supported production deployments and incident fixes in an Agile/Scrum environment.

Project Engineer / Officer at Greenko Group (2017-05 – 2021-06)

• Managed and maintained structured operational records, inspection logs, and quality reports — building strong foundations in data documentation, traceability, and process standardization.
• Coordinated cross-functional project teams and tracked deliverables, timelines, and KPIs — experience directly transferable to Agile sprint planning and stakeholder reporting in data engineering.
• Worked with large volumes of equipment and telemetry data to monitor performance, identify anomalies, and support operational decision-making.

Bachelor of Technology in Electrical & Electronics Engineering – Jawaharlal Nehru Technological University Hyderabad (JNTUH) (2013)