Coffeebeans.io - Lead Data Engineer - Java/Python

Key Responsibilities:

• Design, develop, and maintain efficient, reliable, and scalable data pipelines using modern data engineering tools and best practices.
• Build and optimize data flows between internal and external data sources and destinations.
• Implement data quality checks, monitoring, and alerting mechanisms.
• Write clean, efficient, maintainable, and production-ready code.
• Follow coding standards and engineering best practices.
• Participate in code reviews and contribute to technical documentation.
• Troubleshoot and resolve data pipeline failures, data quality issues, and performance bottlenecks.
• Collaborate with cross-functional teams to support analytics and data-driven business initiatives.

Required Technical Skills:

• Strong proficiency in SQL and data modelling.
• Hands-on experience with Python or Java.
• Proven experience with Apache Spark for large-scale data processing.
• Strong experience with Apache Airflow for workflow orchestration.
• Experience with modern data platforms and cloud data warehouses, particularly Databricks and/or Snowflake.
• Experience with at least one major cloud platform: AWS, GCP, or Azure.
• Strong understanding of data engineering concepts, ETL processes, and data warehousing principles.
• Good understanding of scalable data pipeline architecture and performance optimization.

Good to Have:

• Databricks certification.
• Snowflake certification.
• Experience working across multiple cloud platforms.
• Exposure to modern data engineering and analytics ecosystems.