Data Engineer - Python/Azure Databricks

Key Responsibilities:

• Architect and implement enterprise-grade Lakehouse solutions using Databricks.
• Build scalable batch and real-time data pipelines using PySpark and Spark SQL.
• Develop ETL/ELT pipelines, Delta Lake, Delta Live Tables, Autoloader, and Structured Streaming solutions.
• Design enterprise data warehouse solutions using dimensional modeling and 3NF.
• Implement data governance, security, and data quality using Unity Catalog.
• Drive ML lifecycle management with MLflow and support AI use cases.
• Optimize Spark workloads, implement CI/CD, and manage Azure cloud deployments.
• Lead technical teams, mentor engineers, and collaborate with stakeholders to deliver scalable data solutions.

Requirements:

• 10 years of experience with 5 years in Databricks.
• Strong architecture and solution design expertise.
• Hands-on experience with Databricks, Delta Lake, Delta Live Tables, PySpark, Spark SQL, and Azure Data Factory.
• Experience with Azure, Azure DevOps, CI/CD, and enterprise data platforms.
• Exposure to international (US/UK) client engagements.
• Team leadership and stakeholder management experience.
• Knowledge of AI/ML tools and MLflow is an added advantage.

Benefits:

• Free AWS & Azure certification training.
• Group Medical Insurance of ?5 Lakhs.
• Meal Allowance.
• Opportunity to work on enterprise-scale cloud and AI-driven data platforms.