Vishal Sharma

Experienced in designing high-performance REST APIs, implementing authentication using Spring Security JWT, and resolving production issues efficiently. Skilled in Azure Blob Storage, pipeline monitoring, logging, and troubleshooting data workflows. Passionate about building scalable and reliable cloud-based data engineering solutions.

Quick learner and proficient in solving technical issues in the project. Skilled in debugging and resolving production issues, improving system reliability through structured logging, monitoring, and error-handling. Experienced in Agile methodologies, collaborating with cross-functional teams to deliver business-focused and scalable solutions.

Systems Engineer at Tata Consultancy Services (2022-08 – Present)

Worked on Azure Data Lake Storage and ETL pipeline design for data ingestion and transformation. Managed data workflows, developed Databricks notebooks, and ensured pipeline reliability and security.

• Worked with Azure Data Lake Storage (ADLS) for storing structured and unstructured data, implementing efficient data partitioning strategies.
• Designed and implemented ETL pipelines using Azure Data Factory for data ingestion and transformation.
• Created and managed data workflows to move data between Azure Storage Accounts and Blob Storage.
• Developed Databricks notebooks using Python for data cleansing and transformation.
• Improved data processing efficiency by automating ingestion pipelines.
• Monitored and troubleshot pipeline failures using Azure monitoring tools.
• Collaborated with cross-functional teams to understand business data requirements.
• Implemented a snapshot-based data retention strategy, generating monthly historical datasets retained for 18 months.
• Ingested 50+ JSON files weekly (5–10 GB per load) using Azure Data Factory and Azure Databricks.
• Reduced data ingestion failures by 35% through improved validation checks and monitoring mechanisms.
• Achieved 99.9% pipeline reliability for Azure Databricks workloads through proactive monitoring and optimization.
• Ensured secure data access using Azure Active Directory groups and Azure Key Vault with zero security incidents.
• Maintained source code and version control using Git and GitHub.
• Collaborated with cross-functional teams to translate business requirements into technical solutions.

Masters of Computer Application in Software Engineering – Guru Gobind Singh Indraprastha University (2019-09 – 2022-05)

Bachelor of Computer Application – Guru Gobind Singh Indraprastha University (2016-09 – 2019-05)