Divya Yaramala

Data Engineer with 5+ years of experience designing and optimizing cloud data platforms for retail and enterprise networking organizations. Designed and delivered batch and real-time pipelines on AWS and Azure using Python, SQL, PySpark, Databricks, Snowflake, Airflow, Kafka, and dbt to power enterprise analytics, BI, and machine learning. Owned solutions end to end from architecture through production support, improving pipeline performance, strengthening data quality and governance, and reducing cloud spend through automation.

Data Engineer at Walmart Global Tech (2024-10 – Present)

• Architected and optimized PySpark pipelines on Azure Databricks processing retail, inventory, and customer data, cutting end-to-end execution time by 45% and speeding up data availability for forecasting and merchandising.
• Built scalable ELT frameworks with Azure Data Factory, Snowflake, and Apache Airflow to integrate 120+ enterprise datasets, improving pipeline reliability and maintainability across business domains.
• Engineered near real-time inventory streaming using Apache Kafka and Spark Structured Streaming, cutting data latency from hours to minutes and enabling faster supply chain decisions.
• Designed dimensional data models and Snowflake data marts supporting executive reporting, financial analytics, and self-service BI for cross-functional teams.
• Built automated data quality validation and reconciliation frameworks in Python and SQL, cutting production data discrepancies by 60%.
• Delivered governed, analytics-ready datasets for demand forecasting and AI-driven reporting, improving feature engineering efficiency for downstream ML workloads.
• Developed resilient Airflow orchestration with dependency management, automated recovery, and alerting, reducing operational support effort.
• Standardized CI/CD deployment using Git, Jenkins, Terraform, and Infrastructure as Code, accelerating release cycles and minimizing production failures.
• Tuned Snowflake warehouse performance through clustering and workload balancing, lowering cloud costs while maintaining analytical performance.
• Built and maintained dbt models for core transformation logic in the ELT pipeline, using dbt tests to enforce schema and data quality checks before data reached Snowflake marts.
• Partnered with product managers, data scientists, and business analysts to translate requirements into scalable cloud data solutions.

Data Engineer at Cisco Systems (2020-08 – 2023-12)

• Designed ETL/ELT pipelines in Python, SQL, and Apache Spark integrating ERP, CRM, API, and operational data into centralized analytical platforms.
• Increased large-scale data processing throughput by 40% through Spark partitioning, caching, broadcast joins, and execution plan tuning.
• Migrated on-premises pipelines to AWS Glue, EMR, S3, and Redshift, improving scalability and long-term platform maintainability.
• Built reusable ingestion frameworks for structured, semi-structured, and API-driven sources, cutting new data source onboarding time by 50%.
• Built Airflow orchestration pipelines with automated scheduling and failure recovery, improving production stability.
• Designed dimensional models and Snowflake warehouses supporting finance, sales, and executive reporting.
• Implemented automated data validation and reconciliation frameworks in Python and SQL, strengthening enterprise data quality.
• Tuned complex SQL queries and Spark transformations on multi-terabyte datasets, decreasing execution time by 35%.
• Partnered with business analysts, architects, and QA teams to translate requirements into scalable cloud data solutions.

Master of Science (M.S.) in Information Technology – St. Francis College (2023-08 – 2025-08)