Lead/Senior Data Engineer - ETL/Python

Working Details :

Working Days : Monday - Friday

Job Timing : 9AM - 6PM

Position Details :

Position / Designation : Lead / Senior Data Engineer - Hybrid

Qualification : Bachelor's degree in Computer Science, Information Technology, Engineering, or a related technical field

Years of Experience : 6 to 9 Years

Employment Type : Permanent Full Time

Location: Rajiv Gandhi Salai, Taramani, Chennai

Number of posts: 1

Gender: Male / Female

Joining Date: Immediate to 1 Month

Selection Process :

• Total 3 Technical rounds.
• 2 rounds evaluation with LV and 1 with client.

Job Role &

• Responsibility :

1. Architecture &

• Design :
• Design and architect scalable, high-throughput, and reliable end-to-end data systems (databases, warehouses, and big data platforms) adhering to data governance, privacy, and compliance standards.

2. Lead &

• Technical Delivery :
• Drive the end-to-end technical delivery of complex data initiatives.
• Translate broad business requirements into robust, production-grade technical designs.

3.

Pipeline

Engineering :

• Lead the implementation and optimization of complex ETL/ELT data pipelines using SQL, PySpark, Hive, and Airflow across large-scale distributed environments.

4. Team Leadership &

• Mentorship :
• Provide technical leadership, code reviews, and architectural guidance to junior and mid-level data engineers to enforce best practices and quality standards.

5.

Stakeholder

Management :

• Collaborate with business leadership, product managers, and enterprise architects to align data strategies with overall organization goals.

6. AI &

• Innovation :
• Leverage AI / GenAI tools to streamline pipeline generation, query optimization, automated test suite generation, and metadata management.

Skills :

Primary Skills :

• Advanced PySpark, Advanced SQL, Data Architecture &
• Design, Big Data Ecosystem (Hadoop, HDFS, Hive), Airflow / Workflow Orchestration, Data Warehousing (Snowflake / Redshift / BigQuery).

Secondary Skills :

• Enterprise Cloud Platforms (AWS / GCP / Azure), CI/CD &
• DevOps for Data Platforms, GenAI / AI-driven pipeline automation, Data Governance &
• Quality Frameworks, Streaming (Kafka / Spark Streaming).

Key Skills: Pyspark, Hadoop, HDFS, Hive, Airflow / Workflow Orchestration, Bigquery, Redshift, SQL, Snowflake, Cloud, Spark, ETL, Advanced SQL, Python.