Vinay M Vinu

Azure Data Engineer with 2+ years of experience in designing, developing, and maintaining ETL/ELT data pipelines using Azure Data Factory (ADF), Python, PySpark, and SQL. Experienced in data ingestion, transformation, validation, and loading of large-scale datasets from multiple source systems into cloud-based data platforms. Skilled in pipeline orchestration, workflow automation, performance optimization, and troubleshooting.

Proficient in working with Azure services, relational databases, and big data technologies. Strong analytical and problem-solving abilities with a commitment to delivering high-quality, scalable, and reliable data solutions.

Associate Data Engineer - Ahana Systems and Solutions Pvt. Ltd - Bengaluru, Karnataka

(2024-06)

• Developed and maintained end-to-end ETL/ELT pipelines using Azure Data Factory to ingest, transform, and load data from multiple source systems into cloud-based data warehouses.
• Built dynamic and reusable pipelines using ADF activities such as Copy Activity, Mapping Data Flow, Lookup, ForEach, If Condition, Filter, Execute Pipeline, and Stored Procedure Activity.
• Designed and implemented metadata-driven pipelines, reducing manual effort and improving pipeline reusability across multiple projects.
• Integrated on-premises and cloud data sources using Self-Hosted Integration Runtime for secure and reliable data movement.
• Automated data ingestion processes using parameterized pipelines, dynamic content, variables, and control flow activities.
• Monitored pipeline executions, analyzed failures, and implemented error-handling and retry mechanisms to improve data reliability.
• Developed and optimized data transformation workflows using SQL, improving data processing performance and reducing execution time.
• Scheduled and orchestrated batch data loads using triggers, ensuring timely availability of business-critical data.
• Performed data validation, reconciliation, and quality checks to ensure data accuracy and consistency across systems.
• Implemented incremental loading and Change Data Capture (CDC) strategies to minimize data movement and optimize pipeline performance.
• Loaded data into Azure Data Lake Storage and optimized storage structures for analytics and reporting requirements.
• Developed data transformation solutions using Python and PySpark, processing large-scale datasets efficiently.

Bachelor of Computer Applications - Computer Applications - Kuvempu University (2020 - 2023) Shivamogga

Pre-University Course - Acharya PU College (2018 - 2020) Shivamogga