Divya Kannan

Data Engineer with 3+ years building production-grade platforms on Azure Databricks, PySpark, and Delta Lake — serving 10+ enterprise clients. Expertise in Medallion Architecture, ETL/ELT, dimensional modelling, T-SQL optimization, and data governance. Delivered 30–50% performance gains across pipelines and SQL workloads. Built AI-powered NLP-to-SQL using RAG and FAISS on Azure OpenAI.

Data Engineer at Odessa Technologies (2022-09 – Present)

Data Engineer building production-grade Azure Databricks platforms with Delta Lake for 10+ enterprise clients

• Built Azure Databricks Lakehouse (Medallion: Bronze/Silver/Gold) with Delta Lake — ACID writes, schema evolution & time travel for 10+ enterprise clients
• Developed PySpark pipelines (Spark SQL, DataFrame API, RDD Transformations) processing 50M+ records/day — improved compute efficiency by 30% via dynamic partitioning, broadcast joins & caching
• Replaced full-load batches with CDC + timestamp-based incremental ingestion — cut processing time by 40%, eliminating blocking windows for downstream consumers
• Designed 20+ data models (star/snowflake schema, fact/dim tables, SCD Type 1 & 2) across multiple business domains — improved analytical query performance by 30%
• Engineered T-SQL stored procedures, views, triggers & CTEs for analytics — reduced report execution time by 40% via indexing & query plan optimization
• Integrated REST API data sources for automated ingestion into the Lakehouse — eliminated manual data pulls and improved data freshness across dashboards
• Built and maintained enterprise data warehouse handling 200GB+ data with optimized schemas for high-frequency analytics workloads
• Implemented CI/CD on Azure DevOps for Spark jobs & ETL workflows — cut release effort by 40%, enabling weekly production deploys
• Delivered Power BI & SSRS dashboards — eliminated 40% of manual reporting effort and gave leadership real-time visibility
• Built end-to-end data lineage tracking using Delta Lake audit logs — full visibility across all pipeline layers for governance & root-cause analysis
• Enforced data governance using Delta Lake ACID guarantees, schema enforcement & data quality checks — reduced pipeline failures by 25–30%
• Led pipeline architecture design decisions and mentored junior engineers on PySpark, SQL, and data modelling best practices

MBA in Data Science & Analytics – Jain University (2025-01 – 2027-12)

B.E. in Electronics & Communication Engineering – Sri Ramakrishna Engineering College (2023-05)