Muthuraja Muruganantham

Data Engineer

Talent

LondonMember since September 5, 2026

PythonSQLAWS GlueAmazon AthenaAmazon RedshiftAWS IAMCloudWatchPySparkSpark SQLParquetDelta LakeApache IcebergGitGitHub ActionsCI/CDPartitioningData quality

Hire this person

Send a job offer directly to this candidate

PythonSQLAWS GlueAmazon AthenaAmazon Redshift

About

Data Engineer with 3+ years' experience building production ETL/ELT pipelines and cloud data lake architectures using Python, PySpark, Spark SQL, AWS Glue, S3, Athena, and Redshift, processing 10M+ records daily. MSc Computer Science from Coventry University (2026), including a benchmarking study of Parquet, Delta Lake, and Iceberg across 84M+ rows. Eligible for the UK Graduate visa from November 2026, with full UK work rights and no sponsorship required for 2 years.

Experience

Data Engineer at Tata Consultancy Services (2022-07 – 2025-09)

Designed and delivered scalable ETL/ELT pipelines using Python, PySpark and AWS Glue to ingest, transform and load large-scale datasets from multiple sources into S3-based data lakes and analytics platforms.
Processed 10M+ records daily using PySpark, implementing joins, aggregations, window functions, and Spark optimisation techniques to improve pipeline processing efficiency and reduce run time.
Optimised analytical workloads using Amazon Athena and Amazon Redshift through partitioning, predicate pushdown, and Parquet-based storage, reducing query execution time and data scan costs by 30%.
Provisioned and configured EC2 instances with appropriate security groups and IAM roles to support pipeline execution and testing environments, ensuring secure, repeatable infrastructure setup.
Built an S3 event-triggered Lambda function to validate incoming file availability and schema, invoking the Glue ETL job on success or alerting via SNS on validation failure.
Implemented automated data quality checks, schema validation, null checks, and duplicate detection, reducing manual validation effort by 50% and improving production data accuracy.
Used AWS IAM, CloudWatch, Git, and CI/CD pipeline automation (GitHub Actions) to support secure deployments and maintain reliable production pipelines within Agile delivery cycles, collaborating closely with analysts and data scientists on production-ready solutions.