Lead Azure Data Engineer - ETL/Python

Job Summary :

Lead Azure Data Engineer with 5 years of experience in designing and building scalable data lakes, data warehouses, and ETL pipelines using Azure Data Factory, Azure Databricks, SQL, and Python.

Responsibilities :

• Develop, implement, and maintain the information data lake and use insight platforms to enable decision support systems.
• Partner with business and customers to understand data and reporting requirements.
• Bring data sets together to answer business questions and drive growth.
• Design ETLs to ingest data into the data warehouse and data lake, as well as end-user facing reporting applications.
• Collaborate with business customers and development teams to define requirements and deliver flexible, scalable, end-to-end solutions.
• Work with new technologies while driving business intelligence solutions end-to-end: business requirements, data modelling, ETL, metadata, reporting, and dashboarding.
• Apply expertise in the design, creation, management, and business use of large datasets.
• Define SLA for all areas of ownership.

Tech Stack & Requirements:

• Experience with Data modelling, SQL, ETL, Data Warehousing and Data lakes.
• Experience in writing SQL and Python scripts.
• Experience in building dashboards and reports using Microsoft Power BI.
• Experience in transforming data using Azure Data Factory.
• Experience working with Spark and Azure Databricks.
• Ability to balance and prioritize multiple conflicting requirements with high attention to detail.
• Strong verbal/written communication & data presentation skills.
• Familiarity with scripting languages such as Python, PySpark, or JavaScript.
• Knowledge of Azure Databricks and MPP databases.
• Exposure to NoSQL databases (such as Cosmos DB, MongoDB).
• Experience following Agile processes.
• Experience working with distributed version control systems like Git.
• Experience with Mulesoft/Kingswaysoft/Marketo.