Harini Abburi

Results-driven Senior Data Engineer with 7+ years of experience designing, building, and operating enterprise-scale data pipelines, cloud data platforms, and real-time streaming systems across Canadian financial services and insurtech. Deep expertise in Python, PySpark, SQL, AWS (S3, Lambda, Glue, Kinesis), and Snowflake, with strong proficiency in DBT, Airflow/Astronomer, Kafka, and Medallion Architecture. Proven track record delivering data solutions in regulated environments with adherence to PIPEDA data privacy requirements and FSRA insurance regulatory standards.

Skilled at architecting event-driven ETL/ELT pipelines, implementing CI/CD automation with GitHub Actions, and enabling data-driven decision-making across underwriting, risk analytics, and insurance pricing domains. Currently based in Brampton, Ontario, and actively seeking Senior Data Engineer roles in the Ontario market.

Senior Data Engineer - Foxquilt - Toronto, Ontario, Canada

(2022-04)

• Architected enterprise-scale commercial insurance data ingestion pipelines using AWS and Snowflake, consolidating policyholder, quote, and claims data from 15+ source systems to enable unified underwriting analytics.
• Designed and implemented Medallion Architecture (Bronze/Silver/Gold) in Snowflake and S3, organizing raw policy event data, curated risk features, and business-ready underwriting metrics — reducing time-to-insight for the actuarial and product teams by 60%
• Built event-driven streaming ingestion pipelines using Kafka and AWS Lambda to capture real-time policy bind, quote, and cancellation events from Foxden microservices, enabling live rate adjustment signals and reducing data latency from hours to under 3 minutes
• Developed 20+ analytics dashboards surfacing key insurtech KPIs — loss ratio, quote-to-bind conversion, premium written by business class, and partner channel performance — empowering underwriting, product, and executive teams to make data-driven pricing decisions
• Implemented CI/CD pipelines for data workflows using GitHub Actions and AWS CodePipeline, automating testing and deployment of Snowflake DBT models and Lambda-based ingestion jobs, reducing release cycle time by 65% and eliminating manual deployment errors
• Led data platform build-out to support Foxquilt’s US market expansion and Markel partnership, designing cross-border data schemas to unify Canadian and US policy data under a single Snowflake warehouse while maintaining regulatory separation between jurisdictions
• Developed DBT transformation models to standardize risk signal features used by Foxquilt’s ML underwriting algorithms, ensuring consistent feature definitions across the actuarial rating engine and analytics layer, and reducing data discrepancies in pricing decisions by 35%
• Designed and deployed Airflow DAGs on Astronomer to orchestrate nightly ingestion, DBT transformation, and data quality workflows across Foxden API sources, third-party partner feeds (Munich Re, Markel), and internal Snowflake layers
• Optimized AWS Lambda concurrency, S3 storage tiering, and Snowflake virtual warehouse auto-suspend settings, reducing cloud infrastructure spend by 30% while supporting a 3x increase in daily policy event volume driven by new partner onboarding
• Collaborated with Legal and Compliance teams to embed PIPEDA data privacy controls and FSRA regulatory requirements directly into pipeline design, ensuring policyholder PII was masked at ingestion, audit-logged at transformation, and retained per Ontario insurance data governance standards

Senior Data Engineer - National Bank - Toronto, Ontario

(2021-01 - 2022-03)

• Developed end-to-end data ingestion pipelines on Azure HDInsight Spark using Azure Data Factory, processing 500GB+ daily from heterogeneous sources (MySQL, Cassandra, Blob Storage) to Azure Synapse Analytics
• Migrated 10+ legacy data sets to Databricks Spark platform, reducing processing time from 6 hours to 45 minutes through distributed computing optimization and efficient data partitioning strategies
• Engineered real-time streaming analytics solution using Spark Streaming and Azure Data Lake Gen2, enabling fraud detection with <5 minute latency for banking transactions, improving detection accuracy by 30%
• Built 25+ Power BI dashboards and SSRS reports consumed by 100+ business stakeholders, providing real-time insights into customer behavior, loan performance, and risk management metrics
• Implemented CI/CD pipelines for Azure Data Factory using ARM Templates and Azure DevOps, reducing deployment time by 70% and eliminating configuration drift across environments
• Optimized Azure Kubernetes Service (AKS) cluster configurations for Spark workloads, achieving 30% cost reduction through intelligent auto-scaling and resource allocation
• Configured Azure Logic Apps workflows to schedule and automate batch jobs, integrating 15+ enterprise applications and reducing manual intervention by 80%
• Enhanced analytical query performance by migrating log storage from Cassandra to Azure SQL Data Warehouse, achieving 3x faster queries through columnstore indexes and optimized table design
• Collaborated with cross-functional teams using Azure DevOps for sprint planning, code reviews, and continuous integration processes
• Created Databricks notebooks to streamline and curate data for various business use cases, mounting Blob Storage for seamless data access

Data Engineer - Pineapple Financials Inc - Toronto, Canada

(2019-10 - 2020-12)

• Built distributed data processing jobs using Apache Spark and SQL for large-scale transformations across multi-source ingestion pipelines feeding Snowflake and cloud storage layers.
• Designed ETL pipelines integrating relational (SQL Server) and cloud data sources, implementing Medallion Architecture (Bronze/Silver/Gold) for clean separation of raw, curated, and business-ready data.
• Developed Kafka-based streaming ingestion pipelines to capture CDC (Change Data Capture) events, enabling near-real-time analytics for operational dashboards.
• Created Apache Airflow DAGs to orchestrate multi-step ingestion, transformation, and data quality workflows, replacing manual scheduling with fully automated dependency-driven pipelines.
• Developed Flask REST APIs to expose aggregated data assets to downstream BI tools and internal data consumers, incorporating token-based authentication and rate limiting.
• Containerized pipeline services with Docker; contributed to Kubernetes deployment manifests and CI/CD pipeline configurations in Jenkins for consistent environment delivery.
• Wrote Python automation scripts and shell scripts for operational tasks; managed deployments and monitoring on Linux/Unix environments.
• Optimized complex SQL queries and Snowflake warehouse sizing, improving performance of large-table analytical queries used in executive reporting.
• Maintained PyTest-based unit and integration test suites; integrated test runs into Jenkins build pipelines to enforce quality gates before production deployments.
• Collaborated with cross-functional teams to translate business requirements into backend data models and scalable pipeline architectures within an Agile delivery model.

Bachelor of Technology - Computer Science - JNTU, India