Skip to main content

Senior Data Engineer

Technology
Taguig, Philippines1 week ago

Job description

Role: Senior Data Engineer

Skill

  • Fluent in Python programming language

  • Proficient in SQL

  • Understanding of OLTP concepts including normalised schema design

  • Understanding of OLAP concepts and data model design

  • Experience leading the designing and building data pipelines, ETL processes, and managing data infrastructure

  • Experience leading workflows automation, managing task dependencies and transformations using Apache Airflow and DBT

  • Experience in real time streaming pipelines like Kafka/pubsub and, Pyspark/dataflows

  • A strong analytical mindset, a passion for resolving complex technical challenges, and the ability to find creative solutions to issues

  • Excellent communication and interpersonal skills, with the ability to lead technical discussions, build consensus, and work effectively with cross-functional teams

  • Deep understanding of cloud migration strategies, best practices, and potential pitfalls

  • Experience with Google Cloud Platform, AWS or Azure including services for data storage, processing, and analytics

  • Experience with cloud deployment and test automation using a CI/CD solution such as Cloud Build or Concourse

  • Familiarity with consumer behaviour data (Google Analytics, Adobe Analytics or Snowplow Analytics) and/or subscribers data

  • Proficient usage of AI to gain working efficiency to focus on higher impact architectural tasks

Experience and Role Responsibilities

  • 5 + Senior Data Engineer

  • Design, develop and maintain data pipelines that extract, load and transform data from various sources into the data warehouse

  • Design and implement data models that align with business requirements and support efficient querying and analysis

  • Ensure the team's orchestration pipelines are reliable, scalable and efficient whilst considering factors including data volume, velocity and variety

  • Work across the data engineering teams, vendors and internal stakeholders to contribute to the continuous improvement of data products, processes and systems.

  • Follow data engineering best practices including coding, DevOps, documentation and metadata standardisations

  • Experience with the management and nature of audience tracking and subscribers datasets

  • Take ownership and accountability for assigned work, including development and end-to-end testing

  • Ability to conduct comprehensive feasibility assessments of architectural and data modelling options, systematically evaluating the trade-offs, advantages, and limitations of each proposed solution

  • Ensure high standard and ongoing quality of Engineering output whilst meeting project timelines

  • Document existing and future solution designs to enhance the team's collective knowledge

Keywords
OrchestrationCodingApache KafkaMetadataApache AirflowDevOpsAirflowPythonSqlApache Http ServerCI/CDBig data

Interested in this role?