Sharnam Kansal

Microsoft-certified Data Engineer with 1.5+ years building scalable ETL/ELT pipelines, cloud data warehouses, and Power BI solutions on Microsoft Fabric, Azure (ADF, Databricks, ADLS Gen2), and AWS, using PySpark, SQL, Delta Lake, and Medallion Architecture.

Software Engineer at MAQ Software (2024-12 – Present)

• ETL/ELT pipelines: Engineered end-to-end ingestion and transformation pipelines in Microsoft Fabric (Lakehouse, Pipelines, Notebooks) using PySpark, Python, and SQL, processing multi-million-row datasets and reducing refresh latency by 40%.
• Dimensional modeling: Architected star-schema models with SCD Type 2 history tracking and delivered Power BI dashboards with optimized DAX measures and Row-Level Security (RLS), enforcing KPI validation for enterprise reporting.
• Automation & observability: Built CI/CD workflows on Azure DevOps with Azure–Fabric MCP Servers for deployment, monitoring, and alerting, eliminating 15+ hrs/week of manual effort.
• Lakehouse ingestion: Built ingestion and orchestration pipelines in Azure Data Factory (ADF) with PySpark transformations in Azure Databricks following Medallion Architecture (Bronze/Silver/Gold), managing curated data in ADLS Gen2 (Delta Lake).
• Incremental & schema evolution: Implemented incremental loads, schema evolution, and merge/upsert patterns on Delta tables, ensuring idempotent and resilient ingestion.
• Performance tuning: Optimized Spark jobs via partitioning, Z-ordering, broadcast joins, and parallelization, reducing pipeline runtime by 35% and lowering compute costs.
• Cloud ingestion: Ingested data from AWS S3 via Databricks (PySpark) pipelines for scalable distributed workflows; provisioned infrastructure as code with Terraform in an Agile/Jira environment.
• Data quality & BI: Built automated KPI validation and reconciliation frameworks with RBAC access controls, and delivered Amazon QuickSight dashboards, reducing reporting defects by 50%.

B.Tech in Information Technology – Netaji Subhas University of Technology (NSUT) (2021 – 2025)

Class XII in CBSE – Sidhhartha Public School (2019 – 2020)