Rowan Hall

Experienced Data Engineer with over 9 years of hands-on expertise designing, building, and optimizing scalable data pipelines and analytics solutions on Azure for the financial services industry. Deep proficiency in Azure Databricks, PySpark, Spark SQL, Delta Lake, Azure Data Lake Storage (ADLS), and Azure Data Factory (ADF), with a proven record of supporting payment processing and payment analytics use cases. Skilled in developing high-volume ETL/ELT workflows, integrating data from payment platforms and banking systems, and implementing robust data quality, governance, and monitoring practices.

Adept at collaborating with business, product, and engineering teams to deliver reliable, high-performance data solutions aligned with Syncredible Solutions' focus on digital transformation and payment analytics.

Senior Data Engineer - Omnicell - Fort Worth, TX (remote)

(2019-09)

• Designed and operated Azure-centric cloud data platforms using Azure Databricks, Azure Data Factory (ADF), Azure Synapse Analytics, Azure Data Lake Storage (ADLS), Azure Blob Storage, Azure SQL Database, Azure Functions, Azure Monitor, Application Insights, Microsoft Purview, Azure Key Vault, Managed Identity, and Azure RBAC to support enterprise analytics, reporting, and ML-ready datasets for payment processing and payment analytics use cases.
• Built and orchestrated scalable ETL/ELT pipelines for payment transaction processing and reporting, delivering hundreds of GB of data daily into SQL Server-based data marts, Azure Data Lake Storage, Azure Synapse Analytics, Snowflake, and downstream BI platforms using PySpark, Spark SQL, Delta Lake, and Databricks, improving data freshness, reporting speed, and analytics availability.
• Designed and enforced production data reliability standards using Azure Monitor, Application Insights, Log Analytics, alerting, data quality validation, SLA/SLO tracking, pipeline health checks, and incident-response workflows to improve trust and availability across production data systems, with a focus on payment datasets.
• Architected secure Azure data access patterns using Managed Identity, Azure RBAC, Azure Key Vault, Private Link, encryption, audit logging, backup strategies, lifecycle policies, and governance best practices to support compliant, high-performance enterprise data access in the financial services and payments domain.
• Built scalable storage and backup strategies using Azure Data Lake Storage (ADLS), Azure Blob Storage, lifecycle management, redundancy options, and archive tiers, supporting large-volume enterprise data while improving retention, recovery, and cloud cost efficiency for payment analytics solutions.
• Designed and scheduled Apache Airflow DAGs to orchestrate Azure, Hive, Pig, Spark, SQL, and batch-processing workflows with dependency management, retries, SLA monitoring, and operational alerting, supporting environments involving Databricks and Delta Lake.
• Optimized Spark and PySpark pipelines on Azure Databricks and Azure Synapse Spark using partitioning, caching, broadcast joins, executor tuning, and optimized transformations, improving processing throughput and reducing job runtimes by approximately 6–9%.
• Built and maintained ingestion pipelines using Azure Data Factory (ADF), Fivetran, Talend, Apache NiFi, Azure Functions, and API-based extraction frameworks, enabling reliable data movement from payment platforms, banking systems, and enterprise applications into Snowflake, Azure SQL, SQL Server, and cloud analytical platforms.
• Implemented incremental models, materializations, and transformation logic using dbt, improving transformation efficiency, resource utilization, and maintainability of analytics pipelines, and supported data modeling best practices for payment analytics.
• Supported integration and optimization of enterprise analytics platforms including Azure Synapse Analytics, Azure SQL Database, Snowflake, SQL Server, Power BI, and Tableau, improving ingestion throughput, schema design, and query performance for large BI datasets and payment analytics reporting.
• Built AI-ready data pipelines to prepare structured and unstructured datasets for Azure OpenAI, Microsoft Foundry / Azure AI Foundry, Azure AI Search, Azure Machine Learning, Azure AI Document Intelligence, and Azure AI Language, supporting enterprise search, document intelligence, NLP enrichment, ML feature preparation, and RAG-style use cases aligned with payment analytics.
• Developed document-processing and search-ready datasets using Azure AI Document Intelligence, Azure AI Language, Azure AI Search, OCR workflows, metadata extraction, entity recognition, and vector-ready text preparation to support claims triage, compliance review, enterprise search, and audit-ready reporting; supported workflows in the payments domain.
• Automated deployment and execution of batch, benchmark, and data-processing workloads using Docker, Kubernetes, Azure DevOps, Git-based CI/CD pipelines, Terraform, and ARM/Bicep-style infrastructure patterns, supporting environments involving Databricks and payment analytics solutions.
• Built Python-based data extraction and API frameworks using Flask, Django, Redis, Elasticsearch, MongoDB, PostgreSQL, Snowflake, Azure Functions, and Azure SQL, enabling high-throughput data access, log aggregation, and operational analytics for payment processing and reporting use cases.
• Collaborated with engineering, analytics, ML, product, and business stakeholders to define Azure data platform standards, review designs, improve code quality, and deliver reliable data products for reporting, analytics, AI, and ML use cases, with a focus on payment analytics and scalable data pipelines.

Senior Data Engineer - Gainwell Technologies - Salt Lake City, UT

(2016-01 - 2019-08)

• Built and optimized batch data processing pipelines using PySpark, Spark SQL, Azure Databricks, and Databricks, improving job execution time by approximately 8–9% through partitioning, caching, and optimized joins.
• Implemented near-real-time ingestion pipelines using Apache Kafka, Azure Event Hubs-style streaming patterns, and Apache NiFi, improving data delivery latency and downstream availability for analytics and operational use cases.
• Designed and implemented data models and schemas for Azure Data Lake Storage, improving query performance and reducing retrieval time.
• Automated ETL workflows using Talend, enabling reliable data movement, validation, and reconciliation between MongoDB and SQL Server.
• Used Apache NiFi for ingestion and flow-based processing and Apache Airflow for scheduling, orchestration, retries, and dependency management.
• Developed and maintained analytical data stores using Azure Synapse-style data warehousing patterns, Google BigQuery, Amazon Redshift, and SQL Server, improving dashboard query performance by approximately 8–9%.
• Migrated legacy ETL workflows toward cloud-native data services, including Azure Data Lake Storage, Azure Functions, Azure Data Factory-style pipelines, Azure SQL Database, AWS S3, AWS Glue, AWS Lambda, and BigQuery, cutting batch processing time by approximately 60% and enabling near-real-time analytics access.
• Implemented backup, replication, failover, and recovery strategies for cloud data systems, improving reliability and operational resilience.
• Implemented log indexing and search using Elasticsearch / Azure AI Search-style architectures, improving diagnostics, debugging workflows, and operational visibility.
• Developed and maintained Python-based APIs using Flask, SQLAlchemy, and PostgreSQL, supporting internal data services and application integrations.
• Built RESTful backend services using Node.js and PHP, supporting high-traffic APIs and internal data access workflows.

Software Engineer (Data & Web) - Surescripts LLC - Arlington, VA

(2014-11 - 2015-12)

• Automated and managed cloud-based deployments using early Infrastructure-as-Code practices with CloudFormation and Azure-style deployment patterns, improving environment consistency and deployment repeatability.
• Scaled Python-based web applications on cloud infrastructure, including AWS EC2, Elastic Beanstalk, and Azure App Service-style environments, supporting growing user traffic and improving application availability.
• Optimized cloud storage and retrieval workflows, supporting faster access to application and reporting data.
• Built Python programs to retrieve, process, and analyze data from cloud-hosted systems, supporting operational reporting and analytics workflows.
• Automated machine data ingestion and reporting using Python, SQL, and Pandas, reducing manual effort and improving reporting accuracy.
• Developed web applications using Python, Django, AngularJS, and Bootstrap, managing the full development lifecycle for multiple client projects.
• Supported application health using custom monitoring scripts, log analysis, and alerting to improve issue detection and operational stability.

Master's degree - Computer Science - Georgia Institute of Technology (2016-08 - 2018-05)

Bachelor's degree - Computer Science - Emory University (2010-09 - 2014-06)