Opis stanowiska
Description
We’re looking for a Mid Data Engineer to take ownership of our data lake. You'll own the company's data lake, managing millions of financial events daily (bets, wallet movements, and live odds) across 12M+ active users. You'll define how data is ingested, stored, retained, and governed across S3 and Snowflake/Databricks, ensuring accurate, reconciled, and reliable data for analytics, finance, and regulatory needs.Location: Kraków, Poland. Hybrid — 2 days per week from the office.
Key Responsibilities:
- Own lakehouse architecture, including Bronze/Silver/Gold layers, Iceberg/Delta tables, and schema evolution.
- Build CDC streaming pipelines using Kafka, Debezium, or similar technologies, handling late and duplicate events.
- Optimize data storage and query performance through partitioning, compaction, and file sizing.
- Manage data retention, archival, regulatory requirements, and GDPR deletion.
- Ensure data accuracy through freshness SLAs, drift detection, and reconciliation with wallet and ledger systems.
- Own data governance, including cataloging, lineage, access control, PII masking, encryption, and audit trails.
- Monitor ingestion health, data quality, and cloud storage/compute costs.
- 2 years of experience in Data Engineering, with proven ownership of large-scale data lake or lakehouse architectures.
- Production experience with cloud storage (S3/GCS) and Snowflake, Databricks, or BigQuery.
- Hands-on experience with open table formats (Iceberg, Delta, or Hudi).
- Strong understanding of partitioning, file layout, and query optimization at terabyte scale.
- Experience with CDC and streaming ingestion (Kafka, Debezium, or equivalent).
- Strong SQL and data modeling skills, with a solid understanding of relational databases and financial ledger systems.
- Experience with data governance, including catalogs, lineage, access control, PII, and retention.
- Python and workflow orchestration experience (Airflow, Dagster, or similar).
- Hands-on experience building and deploying AI agents and LLM-powered workflows to automate complex processes at scale.
- Experience in fintech, iGaming, or other regulated, audit-heavy industries.
Keywords
data-lakehousecenters-for-disease-control-and-prevention-cdcchange-data-capturekafkadebeziumtrade-shows-eventsdistribution-and-storagedata-storagedata-managementdata-retentiongeneral-data-protection-regulation-gdprdriftdata-governanceaccess-controlpersonally-identifiable-information-piidata-encryptionauditsenvironment-health-and-safety-hssedata-qualitydata-engineering
Interesuje Cię ta oferta?