Vinit Kumar

Data Engineer with 4 years of experience in building large-scale data pipelines, ETL processes, and data warehouse solutions. Utilized technologies like Kafka, PySpark, SQL, Databricks and Google Cloud Platform (GCP) services like BigQuery, Cloud Composer, Dataproc to develop multi-terabyte scalable big data solutions.

Data Engineer-Banking Data Platform (Lloyds Banking Group) at Tata Consultancy Services (2021-10 – Present)

• Owned and delivered end-to-end analytics modules as part of large consulting case teams, designing and implementing scalable data pipelines for streaming and batch workloads on Google Cloud Platform (GCP).
• Designed, developed, and maintained robust ETL/ELT pipelines ingesting 20M+ events per day from heterogeneous sources using Confluent Kafka, PySpark, Cloud Dataproc, and BigQuery, enabling analytics-ready datasets for downstream consumers.
• Led the modernization of a legacy Apache Storm-based ETL architecture, migrating to event-driven architecture using Kafka for real-time data processing, significantly improving latency (20%), scalability, fault tolerance, recovery time, and operational simplicity.
• Implemented Cloud Composer (Apache Airflow) workflows for orchestration, managing Spark job lifecycles, dependency handling, SLA monitoring, incremental data loads, and automated failure recovery.
• Optimized PySpark and SQL workloads by addressing data skew, inefficient joins, and excessive shuffles, significantly improving pipeline performance by 30% and cost efficiency.
• Contributed to CI/CD and DevOps practices, using GitHub and Jenkins for version control, automated builds, and deployments; followed Agile methodologies and software engineering best practices.
• Built analytics-ready data models using dbt on BigQuery, creating well-documented staging, intermediate, and mart layers with automated tests, freshness checks, and lineage.
• Partnered with product, analytics, and backend teams to deliver self-serve analytics, powering dashboards, experimentation, and business metrics directly from BigQuery.

Bachelor of Technology in Electronics and Communication Engineering – Guru Gobind Singh Indraprastha University, Delhi (2017-08 – 2021-09)