Saravanakumar S

Data Engineer with nearly 3 years of experience building and maintaining ETL/ELT data pipelines for analytics and reporting. Strong experience with AWS Glue, Amazon S3, AWS Lambda, Step Functions, Amazon Athena, and Amazon Redshift, along with Python, SQL, PySpark, and Spark SQL for data ingestion and processing. Hands-on experience with Snowflake, Snowpipe, and dbt for data loading, transformation, data modeling, and analytics.

Experienced in using Databricks for large-scale data processing and Apache Airflow for workflow orchestration, scheduling, and monitoring. Skilled in SQL optimization, data quality, validation, and performance tuning, delivering reliable data solutions for BI and analytics teams.

Data Engineer - Tech Mahindra - Chennai

(2024-03)

Client: Leading Public Sector Bank

• Investment Banking Data & Analytics Platform using AWS and Snowflake
• Built and maintained ETL/ELT data pipelines using AWS Glue, Amazon S3, Python, PySpark, and SQL to ingest and process data from multiple source systems.
• Implemented a Medallion Architecture (Bronze, Silver, and Gold) using Databricks, Apache Spark, and Delta Lake to cleanse, deduplicate, standardize, and enrich banking datasets, loading data into Snowflake.
• Developed Snowflake data loading and incremental processing workflows using Snowpipe, Streams, and Tasks, and used dbt to build and maintain SQL-based data models.
• Implemented reusable SQL and Spark SQL validation checks for record counts, duplicate records, NULL values, and schema validation, along with source-to-target reconciliation to ensure data quality.
• Optimized AWS Glue and PySpark processing, along with Snowflake SQL queries and virtual warehouse usage, through partitioning and query tuning to improve performance and cost efficiency.
• Built and maintained Apache Airflow DAGs for pipeline scheduling, dependency management, monitoring, retries, and failure handling, using Git/GitHub and CI/CD for controlled deployments.
• Legacy Database Migration to AWS and Snowflake
• Analyzed legacy database schemas and source systems to plan migration to AWS and Snowflake.
• Used AWS DMS and Amazon S3 to migrate and stage historical and incremental data from legacy databases.
• Developed AWS Glue jobs using PySpark, Python, and SQL for data extraction, cleansing, transformation, and validation.
• Worked with Snowflake to load and manage migrated data, including Snowpipe for automated data loading.
• Used dbt and Snowflake SQL to build transformation models and prepare curated datasets after migration.
• Used AWS Lambda, Step Functions, and Apache Airflow to automate migration workflows. Performed data validation, reconciliation, and post-migration testing to ensure data accuracy and completeness.

Bachelor of Engineering - Computer Science and Engineering - Government College of Engineering, Salem (2019 - 2023)