Data Engineer - Python/ETL

Job Description:

Responsibilities:

• Design, build, and optimize ETL/ELT pipelines for batch and streaming data.
• Develop and maintain data warehouses and data lakes (e.g., Snowflake, BigQuery, Redshift).
• Collaborate with data scientists, analysts, and product teams to understand data needs.
• Ensure data quality, reliability, and observability across pipelines.
• Build and manage orchestration workflows (e.g., Airflow, Prefect, Dagster).
• Implement best practices for data modeling, partitioning, and performance tuning.
• Monitor and troubleshoot data infrastructure issues in production.

Requirements:

• 5+ years of experience in data engineering or a related role.
• Proficiency in SQL and Python (or Scala/Java).
• Experience with cloud platforms (AWS, GCP, or Azure).
• Hands-on with big data tools: Spark, Kafka, dbt, or similar.
• Familiarity with data warehousing concepts and dimensional modeling.
• Strong understanding of CI/CD and version control (Git).

Nice to Have:

• Experience with real-time/streaming pipelines (Kafka, Flink, Kinesis).
• Exposure to ML pipelines or MLOps tooling.
• Knowledge of data governance and data catalog tools (e.g., Atlan, Alation).