Azure Data Engineer

Job Title : Azure Data Engineer (7 Years Experience)

Location : Hyderabad - Hybrid

Experience : 7 Years

Employment Type : Full-Time

Job Summary:

We are looking for an experienced Azure Data Engineer with 7 years of expertise in designing, developing, and maintaining scalable data solutions on Microsoft Azure. The ideal candidate should have strong experience with Azure data services, ETL/ELT pipelines, cloud data warehousing, big data processing, and SQL/Python development. The role involves collaborating with cross-functional teams to build reliable, secure, and high-performance data platforms.

Key Responsibilities :

• Design, develop, and maintain scalable data pipelines using Azure data services.
• Build and optimize ETL/ELT processes using Azure Data Factory (ADF).
• Develop data integration solutions using Azure Databricks, PySpark, and Spark SQL.
• Design and implement data lakes using Azure Data Lake Storage Gen2 (ADLS).
• Develop and maintain cloud-based data warehouse solutions using Azure Synapse Analytics.
• Create and optimize SQL queries, stored procedures, and database objects.
• Work with structured, semi-structured, and unstructured data from multiple sources.
• Implement data quality, validation, governance, and security best practices.
• Optimize data processing performance and troubleshoot pipeline failures.
• Automate deployments using Azure DevOps and CI/CD pipelines.
• Monitor Azure resources and ensure high availability and performance.
• Collaborate with business analysts, data scientists, and application teams to deliver data solutions.
• Prepare technical documentation and provide production support.

Required Skills :

1. Azure Services :

• Azure Data Factory (ADF)
• Azure Databricks
• Azure Synapse Analytics
• Azure Data Lake Storage Gen2 (ADLS Gen2)
• Azure SQL Database, Azure Blob Storage, Azure Key Vault, Azure Monitor / Log Analytics.

2. Programming & Databases :

• SQL (Advanced)
• Python
• PySpark
• Spark SQL, T-SQL.

3. Big Data Technologies :

• Apache Spark
• Delta Lake
• Data Lake Architecture
• Data Warehousing Concepts.

4. DevOps & Version Control :

• Azure DevOps, Git, CI/CD Pipelines.

5. Methodology :

• Agile/Scrum methodology.

Mandatory Qualifications :

• Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
• 7 years of experience in Data Engineering.
• Minimum 4 years of hands-on experience with Microsoft Azure Data Platform.
• Strong experience with Azure Data Factory and Azure Databricks.
• Hands-on experience with Azure Synapse Analytics.
• Strong SQL and Python programming skills.
• Experience in building enterprise-scale ETL/ELT pipelines.
• Experience working with Azure Data Lake Storage.
• Good understanding of cloud architecture and distributed data processing.

Preferred Qualifications :

• Microsoft Azure Data Engineer Associate certification.
• Experience with Microsoft Fabric.
• Experience with Delta Lake and Lakehouse architecture.
• Knowledge of Power BI integration.
• Experience with Kafka or Event Hubs for streaming data.
• Exposure to Snowflake or other cloud data warehouses.
• Experience working in banking, healthcare, insurance, or retail domains.

Nice to Have :

• Microsoft Fabric
• Azure Functions
• Logic Apps
• Azure Event Hubs
• Azure Stream Analytics
• Docker
• Kubernetes
• Terraform, Purview, Power BI.

Soft Skills :

• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management.
• Ability to work independently and in a collaborative team environment.
• Strong documentation and presentation skills.
• Ability to mentor junior engineers and participate in technical design discussions.

Experience Requirements :

• Total IT Experience : 7 Years
• Azure Data Engineering : 4 Years
• Azure Data Factory : 4 Years
• Azure Databricks & PySpark : 3 Years
• Azure Synapse Analytics: 2 Years
• SQL Development : 5 Years
• Python Development : 3 Years