Aidan S

Senior Data Engineer with 9+ years of experience designing and delivering scalable data platforms, high-volume pipelines, and cloud-native data solutions across healthcare, fintech, and SaaS environments. Expertise in batch and real-time processing, cloud data warehouses, lakehouse architecture, and modern ELT platforms supporting analytics, operational reporting, and machine learning. Hands-on expertise across Python, SQL, Spark, Kafka, Airflow, dbt, Snowflake, Databricks, AWS, Azure, and GCP, with strong capabilities in data architecture, modeling, distributed processing, data quality, observability, and pipeline reliability.

Proven technical leader who drives architecture decisions, mentors engineers, establishes engineering standards, and leads modernization initiatives while partnering with engineering, product, analytics, and data science teams to deliver reliable, high-impact data solutions.

Lead Data Engineer at CareRev (2022-04 – Present)

• Lead the architecture and evolution of large-scale cloud data platforms on AWS, designing scalable ingestion, processing, storage, and serving layers that process 70+ TB of data daily and support analytics, operational intelligence, and ML workloads for 1,000+ users.
• Modernized enterprise data pipelines by migrating 200+ legacy workflows to a standardized dbt and Apache Airflow-based ELT architecture, reducing infrastructure costs by 30%, improving pipeline reliability by 60%, and establishing reusable patterns for orchestration, testing, deployment, and data transformation.
• Architect and optimize batch and real-time data processing platforms using Apache Spark, PySpark, Kafka, and Spark Streaming, enabling high-throughput and low-latency data products for fraud detection, customer intelligence, predictive analytics, and operational reporting.
• Design scalable data warehouse, lake, and lakehouse architectures, applying dimensional modeling, partitioning, indexing, CDC, query optimization, and performance-tuning strategies to improve data accessibility, processing efficiency, and downstream analytics performance.
• Establish data quality, governance, lineage, and observability standards using Monte Carlo, Great Expectations, Prometheus, and Grafana, implementing monitoring for data freshness, pipeline failures, quality checks, and SLA/SLO compliance across critical production workloads.
• Partner with Data Science, Product, Analytics, and business stakeholders to deliver governed and ML-ready data products, including trusted datasets and feature pipelines supporting machine learning, customer analytics, fraud detection, and predictive decision-making.
• Provide technical leadership across the data engineering organization, driving architecture reviews, technical design decisions, engineering standards, code reviews, and platform modernization initiatives while mentoring engineers and guiding best practices for scalable, reliable, and maintainable data systems.

Senior Data Engineer at Ayar Labs (2019-11 – 2022-04)

• Built and optimized enterprise-scale healthcare data platforms, integrating clinical, claims, patient, and operational data from disparate systems to deliver trusted datasets for analytics, reporting, and downstream applications.
• Developed high-performance batch and streaming pipelines using Python, SQL, Apache Spark, and Kafka, improving data processing efficiency and reducing latency for real-time clinical alerting by 50%.
• Designed scalable data warehouse architectures and analytical data models across Snowflake and Azure Synapse, leveraging dimensional modeling, partitioning, indexing, and query optimization to improve performance for complex analytical workloads.
• Implemented automated data quality and governance frameworks using Great Expectations and Collibra, strengthening validation, metadata management, lineage, and data consistency across regulated healthcare environments.
• Automated data validation, lineage tracking, audit preparation, and compliance processes, reducing manual audit effort by approximately 70% and improving operational efficiency across data workflows.
• Developed reusable engineering patterns for ETL/ELT, data integration, transformation, validation, and pipeline orchestration, improving maintainability and consistency across production data workloads.
• Partnered with data scientists, analysts, product teams, engineers, and healthcare stakeholders to translate complex requirements into scalable data products while supporting HIPAA, HL7, and FHIR compliance and data interoperability standards.

Data Engineer at Clever Inc. (2017-05 – 2019-11)

• Built scalable Python and SQL ETL/ELT pipelines integrating SaaS, CRM, application, and operational data into centralized analytics platforms.
• Designed Star Schema and 3NF data models, improving analytical performance and reducing dashboard response times by up to 10x.
• Implemented Apache Airflow orchestration for ingestion, transformation, validation, and reporting workflows, reducing manual reporting effort by 75%.
• Supported migration of analytical workloads from SQL Server to Amazon Redshift, strengthening cloud-based data warehousing capabilities and scalability.
• Developed API-based data integrations and reusable transformation workflows, improving data quality, reliability, and maintainability across production systems.
• Partnered with business, analytics, and product teams to deliver trusted datasets and self-service reporting solutions that accelerated data-driven decision-making.

Bachelor of Science in Computer Science