We are seeking a skilled GCP Data Engineer to design, develop, and maintain scalable data pipelines and cloud-based data platforms on Google Cloud Platform (GCP). The ideal candidate should have strong experience with BigQuery, Dataflow, Cloud Composer, Dataproc, and Python, along with expertise in ETL/ELT development, data modeling, and cloud-native data engineering solutions. You will collaborate with data architects, analysts, and business stakeholders to build reliable, high-performance data solutions that support analytics and business intelligence initiatives.
Role Description :
- Design, develop, and maintain scalable data pipelines using GCP services.
- Build and optimize ETL/ELT workflows for batch and real-time data processing.
- Develop data solutions using BigQuery, Dataflow, Dataproc, and Cloud Composer (Apache Airflow).
- Ingest data from multiple structured and unstructured data sources.
- Develop and optimize SQL queries for analytics and reporting.
- Implement data transformation and orchestration processes.
- Design and maintain cloud-based data warehouses and data lakes.
- Ensure data quality, governance, security, and compliance across data platforms.
- Optimize data processing jobs for scalability, reliability, and cost efficiency.
- Collaborate with data scientists, analysts, and application teams to support business requirements.
- Monitor pipeline performance and troubleshoot production issues.
- Participate in code reviews and implement engineering best practices.
- Work in Agile environments and contribute to continuous improvement initiatives.
Key Responsibilities :
- Build scalable data pipelines on Google Cloud Platform.
- Develop ETL/ELT processes using GCP services.
- Manage enterprise data warehousing solutions using BigQuery.
- Implement workflow orchestration using Cloud Composer.
- Process large datasets using Dataflow and Dataproc.
- Develop reusable data transformation frameworks.
- Ensure high availability and performance of data platforms.
- Implement data validation and quality checks.
- Support production deployments and ongoing maintenance.
- Optimize cloud resource utilization and cost.
Required Skills :
- 4 - 8 years of experience in Data Engineering.
- Strong hands-on experience with Google Cloud Platform (GCP).
- Expertise in BigQuery for data warehousing and analytics.
- Experience with Cloud Dataflow (Apache Beam).
- Hands-on experience with Cloud Composer (Apache Airflow).
- Experience with Dataproc, Cloud Storage (GCS), and Pub/Sub.
- Strong programming skills in Python.
- Advanced SQL development and query optimization skills.
- Experience building ETL/ELT pipelines.
- Knowledge of data modeling (Star Schema, Snowflake Schema).
- Experience with version control systems such as Git.
- Understanding of CI/CD processes for data engineering.
- Experience working in Agile/Scrum environments.
SysMind