Data Engineer - Python/PostgreSQL

About the Role :

We are looking for an experienced Data Engineer with 6+ years of experience in designing, developing, and managing scalable data pipelines and cloud-based data solutions. The ideal candidate should have strong expertise in Python, Cloud SQL, PostgreSQL, and Cloud Pub/Sub, along with a solid understanding of data engineering principles and cloud technologies.

Key Responsibilities :

• Design, develop, and maintain scalable data pipelines for processing and transforming large volumes of data.
• Develop and manage cloud-based data solutions to support business and analytics requirements.
• Build data ingestion and processing workflows using Python.
• Work extensively with PostgreSQL and Cloud SQL for data storage, processing, and optimization.
• Implement event-driven data pipelines and messaging solutions using Cloud Pub/Sub.
• Perform data extraction, transformation, validation, and loading across various data sources.
• Optimize database queries, data pipelines, and processing workflows for performance and scalability.
• Ensure data quality, accuracy, consistency, and reliability across data platforms.
• Develop monitoring and error-handling mechanisms for data pipelines.
• Collaborate with Data Scientists, Analysts, Software Engineers, and other stakeholders to understand data requirements.
• Troubleshoot data pipeline, database, and production issues.
• Maintain technical documentation related to data pipelines, architecture, and processes.
• Follow best practices for data security, governance, scalability, and cloud infrastructure.

Required Skills :

• 6+ years of professional experience in Data Engineering.
• Strong programming experience with Python.
• Hands-on experience with Cloud SQL and PostgreSQL.
• Experience with Cloud Pub/Sub and event-driven data processing.
• Strong understanding of ETL/ELT processes and data pipeline development.
• Experience working with cloud-based data engineering environments.
• Good understanding of database design, SQL, data modeling, and query optimization.
• Experience with data integration and processing from multiple data sources.
• Strong troubleshooting and problem-solving skills.
• Good understanding of data quality and data governance practices.