Role Overview :
As a Senior Data Engineer, you will serve as a cornerstone of our data architecture team, designing and maintaining robust pipelines that transform raw information into actionable business intelligence. You will collaborate closely with cross-functional teams, including data scientists, product managers, and business stakeholders, to bridge the gap between complex data infrastructure and strategic decision-making. By architecting scalable solutions within the Azure ecosystem, you will directly influence our ability to leverage Artificial Intelligence and advanced analytics, ultimately driving efficiency and innovation across our global operations in Metro cities.
Key Responsibilities :
- Engineer and optimize complex ETL pipelines using PySpark and Python to ensure seamless data ingestion and processing for high-volume Big Data environments.
- Design and implement end-to-end data integration solutions using Azure Data Factory and Microsoft Fabric to streamline data movement across disparate systems.
- Develop high-performance data models and analytical structures within Azure Databricks and Synapse Analytics to support real-time reporting and advanced AI initiatives.
- Collaborate with internal stakeholders to translate business requirements into technical specifications, ensuring data accuracy and accessibility for end-users.
- Maintain rigorous standards for data quality and security, proactively identifying bottlenecks to enhance system performance and reliability.
Required Skillset :
- Demonstrated expertise in building scalable data architectures using PySpark, Python, and SQL, with a deep understanding of Big Data processing frameworks.
- Proven ability to orchestrate complex workflows using Azure Data Factory and manage unified analytics platforms like Microsoft Fabric, Azure Databricks, and Synapse Analytics.
- Strong analytical mindset with the ability to communicate technical concepts to non-technical stakeholders, fostering a collaborative environment across teams.
- A Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field, reflecting a strong foundation in data engineering principles.
- Exceptional adaptability to a hybrid work environment, with the ability to manage projects independently while contributing effectively to a high-performing, distributed team.
- A track record of 5 to 8 years in data engineering roles, showcasing a consistent ability to deliver high-impact data solutions in enterprise-grade environments.
Ushta Te Consultancy Services