Senior Manager—Data Engineer

About the Role

We are looking for a highly skilled Data Engineer to design, build, and optimize our data infrastructure and pipelines. You will work closely with data scientists, analysts, and product/engineering teams to ensure reliable, scalable, and high‑quality data delivery that supports analytics, machine learning, and business decision‑making.

Key Responsibilities

1. Data Pipeline & ETL Development

• Design, build, and maintain scalable ETL/ELT pipelines for batch and real‑time data processing.
• Develop data ingestion frameworks from multiple sources (API, DB, message queues, cloud storage).
• Ensure data quality, validation, and monitoring across the entire pipeline.

2. Data Architecture & Modeling

• Design and optimize data warehouse / data lake architectures.
• Build efficient data models to support BI, analytics, and ML workloads.
• Implement best practices for data partitioning, indexing, and performance tuning.

3. Data Platform Engineering

• Develop and maintain data platform components (workflow orchestration, metadata management, lineage tracking).
• Optimize storage and compute costs in cloud environments.
• Ensure high availability, reliability, and scalability of data systems.

4. Collaboration & Cross‑Functional Support

• Work closely with data scientists to productionize ML features and datasets.
• Partner with engineering teams to integrate data solutions into product systems.
• Support business teams with data accessibility, documentation, and troubleshooting.

5. Governance, Security & Compliance

• Implement data governance standards, including data cataloging, lineage, and access control.
• Ensure compliance with data privacy and security policies.
• Establish monitoring, alerting, and incident response for data pipelines.

Required Qualifications

Technical Skills

• Strong programming skills in Python / Java / Scala.
• Hands‑on experience with SQL and performance tuning.
• Experience with modern data processing frameworks:
• Spark, Flink, Beam, Kafka, Airflow, Dagster, Prefect
• Experience with cloud platforms:
• AWS / GCP / Azure (e.g., S3, Redshift, BigQuery, Snowflake, Databricks).
• Solid understanding of data warehouse / data lake architectures.
• Experience with CI/CD, containerization (Docker), and version control (Git).

Soft Skills

• Strong problem‑solving and analytical thinking.
• Ability to work cross‑functionally with engineering, product, and data teams.
• Good communication skills and documentation habits.

Preferred Qualifications (Nice to Have)

• Experience with machine learning pipelines or feature stores.
• Knowledge of data governance frameworks (e.g., DataHub, Amundsen, Collibra).
• Experience with real‑time streaming (Kafka, Pulsar, Kinesis).
• Familiarity with dbt for data transformation.
• Experience in industries such as e‑commerce, retail, fintech, AI, manufacturing.

Education & Experience

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or related fields.
• 5–8 years of experience in data engineering or similar roles.