Przejdź do treści głównej

Middle Data Engineer/ Data Lakehouse

Technology
4 dni temu
Pełni zdalna

Opis stanowiska

Description

We’re looking for a Mid Data Engineer to take ownership of our data lake. You'll own the company's data lake, managing millions of financial events daily (bets, wallet movements, and live odds) across 12M+ active users. You'll define how data is ingested, stored, retained, and governed across S3 and Snowflake/Databricks, ensuring accurate, reconciled, and reliable data for analytics, finance, and regulatory needs.

Location: Kraków, Poland. Hybrid — 2 days per week from the office.

Key Responsibilities:

  • Own lakehouse architecture, including Bronze/Silver/Gold layers, Iceberg/Delta tables, and schema evolution.
  • Build CDC streaming pipelines using Kafka, Debezium, or similar technologies, handling late and duplicate events.
  • Optimize data storage and query performance through partitioning, compaction, and file sizing.
  • Manage data retention, archival, regulatory requirements, and GDPR deletion.
  • Ensure data accuracy through freshness SLAs, drift detection, and reconciliation with wallet and ledger systems.
  • Own data governance, including cataloging, lineage, access control, PII masking, encryption, and audit trails.
  • Monitor ingestion health, data quality, and cloud storage/compute costs.
Requirements Must-have:
  • 2 years of experience in Data Engineering, with proven ownership of large-scale data lake or lakehouse architectures.
  • Production experience with cloud storage (S3/GCS) and Snowflake, Databricks, or BigQuery.
  • Hands-on experience with open table formats (Iceberg, Delta, or Hudi).
  • Strong understanding of partitioning, file layout, and query optimization at terabyte scale.
  • Experience with CDC and streaming ingestion (Kafka, Debezium, or equivalent).
  • Strong SQL and data modeling skills, with a solid understanding of relational databases and financial ledger systems.
  • Experience with data governance, including catalogs, lineage, access control, PII, and retention.
  • Python and workflow orchestration experience (Airflow, Dagster, or similar).
  • Hands-on experience building and deploying AI agents and LLM-powered workflows to automate complex processes at scale.
Nice to have:
  • Experience in fintech, iGaming, or other regulated, audit-heavy industries.
Keywords
data-lakehousecenters-for-disease-control-and-prevention-cdcchange-data-capturekafkadebeziumtrade-shows-eventsdistribution-and-storagedata-storagedata-managementdata-retentiongeneral-data-protection-regulation-gdprdriftdata-governanceaccess-controlpersonally-identifiable-information-piidata-encryptionauditsenvironment-health-and-safety-hssedata-qualitydata-engineering

Interesuje Cię ta oferta?

Podobne oferty