Azure Data Engineer - Python/SQL

Role: Azure Data Engineer

Job location: Trivandrum (Interested for this location only apply)

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks.
• Build robust ETL/ELT solutions using PySpark and Python for large-scale data processing.
• Implement and manage Unity Catalog for data governance, security, and access management.
• Develop and maintain Delta Live Tables (DLT) pipelines to support reliable and efficient data ingestion and transformation.
• Integrate data from multiple enterprise sources, including TIBCO, On-Premises SQL Server, SharePoint, and Shared Folders.
• Collaborate with business stakeholders and technical teams to understand data requirements and deliver high-quality solutions.
• Implement CI/CD pipelines and automate deployments using Azure DevOps.
• Support Azure cloud infrastructure, release management, and production deployments.
• Optimize data pipelines for performance, scalability, reliability, and cost efficiency.
• Ensure adherence to enterprise security, governance, and compliance standards.
• Troubleshoot production issues and provide ongoing support for data platform operations.

Required Technical Skills:

1.

Azure

Technologies:

• Azure Databricks
• Unity Catalog
• Delta Live Tables (DLT)
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• Azure DevOps (CI/CD, Repos, Pipelines, Release Management)
• Strong Azure Infrastructure and Release Engineering experience

2. Programming:

• Python
• PySpark
• SQL

3. Data Integration:

• Experience working with: TIBCO, On-Premises SQL Server, SharePoint, and Shared Folder/File-based integrations.

Required Experience:

• Azure Databricks: 5+ years
• Azure Data Factory: 5+ years
• Azure DevOps: 5+ years
• Unity Catalog: 2+ years
• Strong experience with enterprise-scale Azure data platform implementations.
• Experience designing secure, scalable, and high-performance data solutions.