Role Overview :
As a Lead Data Engineer, you will spearhead the design, development, and optimization of robust data architectures that serve as the backbone for our analytical and operational decision-making. You will work closely with cross-functional teams, including Data Scientists, Product Managers, and Business Stakeholders, to transform complex raw data into actionable insights. By building scalable, high-performance data pipelines and maintaining reliable data warehouses, you will directly influence business outcomes, ensuring that our data infrastructure remains agile, secure, and capable of supporting rapid organizational growth in a fast-paced metro environment.
Key Responsibilities :
- Architect and maintain end-to-end data pipelines to ensure seamless data flow from diverse sources into our centralized data warehouse for real-time analytics.
- Lead the migration and optimization of legacy data systems to cloud-based environments to enhance system performance and reduce operational latency.
- Collaborate with engineering teams to implement sophisticated data modeling strategies that improve query efficiency and support complex business reporting requirements.
- Design and manage high-throughput streaming architectures using distributed messaging systems to provide stakeholders with timely, accurate data for critical decision-making.
- Mentor junior engineers through code reviews and technical guidance to foster a culture of engineering excellence and best-practice adoption across the data team.
Required Skillset :
- Demonstrated expertise in building and managing large-scale data platforms using Python, Scala, and SQL to solve complex data integration challenges.
- Proven ability to leverage Big Data frameworks such as Spark and Hadoop to process massive datasets efficiently in distributed computing environments.
- Strong proficiency in cloud-native technologies, specifically AWS, with a focus on deploying scalable data solutions that meet stringent security and performance standards.
- Deep understanding of NoSQL and traditional data warehousing architectures, enabling you to select the right storage solutions for varied business use cases.
- Excellent communication skills with the ability to translate technical data requirements into clear, actionable insights for non-technical stakeholders.
- Exceptional problem-solving capabilities and the adaptability to thrive in a hybrid work environment, collaborating effectively across distributed teams.
- A Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field, backed by 5 to 10 years of professional experience in data-intensive roles.
Good Co