Data Engineer - Google Cloud Platform

Description :

Roles and Responsibilities :

• Design, develop, and own scalable batch and near-real-time data pipelines on GCP, including large-scale streaming and batch processing
• Build and optimize ETL/ELT workflows using Cloud Composer and Dataflow for structured and semi-structured data
• Create and manage robust data models to support analytics and reporting use cases
• Optimize BigQuery performance and cost efficiency
• Implement and enforce data quality, governance, security, and compliance standards
• Develop data validation frameworks and quality checks
• Lead data architecture decisions aligned with business and enterprise strategy
• Collaborate with architects, analysts, data scientists, and product teams to ensure data readiness
• Provide production support, troubleshoot issues, and perform root cause analysis
• Drive performance optimization and continuous improvement initiatives

Key Skills :

• Cloud Platform : Google Cloud Platform (GCP) BigQuery, Cloud Composer (Airflow), Dataflow (Apache Beam), Cloud Storage
• Programming : Python, PySpark, Advanced SQL, Java/Scala (preferred)
• Data Engineering : ETL/ELT pipeline development, batch & near real-time processing, metadata-driven frameworks
• Streaming Technologies : Pub/Sub, Kafka
• Data Warehousing : BigQuery (Snowflake experience transferable)
• Workflow Orchestration : Cloud Composer (Airflow)
• DevOps & CI/CD : Git, CI/CD pipelines
• Containers (Optional) : Docker, Kubernetes
• BI & Reporting (Awareness) : Looker, Power BI
• Data Governance : Data quality, security, compliance, and validation frameworks
• Core Competencies : System design, problem-solving, performance optimization