Descrierea postului
Senior Databricks Data Engineer / DWH Developer
Location: Hybrid
Workload: 100% / Full-Time
Languages: English
Position Summary
We are seeking a Senior Databricks Data Engineer / DWH Developer for a 1-year engagement. In this role, you will design, build, and optimize enterprise-scale data pipelines and data warehousing solutions leveraging modern cloud technologies and Delta Lake.
You will work closely with cross-functional data teams to transform complex data streams into reliable, high-performance analytical assets for business decision-making.
Key Responsibilities &
- Deliverables
- ETL Pipeline Development: Architect and maintain robust, scalable ETL/ELT pipelines for batch and real-time processing using PySpark or Scala.
- Lakehouse Implementation: Design and optimize data storage and processing frameworks utilizing Databricks and Delta Lake architectures.
- Data Warehousing &
- Modeling: Build production-grade data models (star schema, snowflake, Data Vault) and optimize query performance across analytical platforms.
- Monitoring: Monitor pipeline stability, optimize Spark job performance, reduce compute costs, and resolve data quality issues.
Required
Qualifications &
- Skills
- Databricks Expertise: Strong practical experience developing production data solutions on the Databricks platform.
- Programming Languages: Proficiency in PySpark and/or Scala, alongside strong advanced SQL skills.
- Data Lake &
- Storage: Deep understanding of Delta Lake transaction layer and storage optimizations (ACID transactions, time travel, Z-Ordering).
Preferred / Nice-to-Have
- Experience with CI/CD for Databricks (Databricks Asset Bundles, Terraform, GitHub Actions).
- Exposure to workflow orchestration tools (Airflow, Azure Data Factory, Databricks Workflows).
- Relevant Databricks or Cloud certifications (e.g., Databricks Certified Data Engineer Professional).
Keywords
OrchestrationScalaSCHEMAApache SparkCloud computingAirflowSqlAcid ProCI / CDBig dataAWSGithubTerraformCI/CD
Te interesează acest rol?