Databricks Engineer with AWS

Overview

At DevsReady, we're looking for a Mid-Senior Databricks Engineer with strong AWS experience to design, build, and optimize scalable data platforms and pipelines.

In this role, you will:

Design and build scalable data pipelines on Databricks running on AWS

Develop and maintain Lakehouse architectures using Delta Lake

Implement batch and streaming data processing solutions with Apache Spark

Ensure data quality, governance, and security across the data platform

Collaborate with business and technical stakeholders to turn raw data into reliable, analytics-ready datasets

Key Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using Databricks, PySpark, and Spark SQL
• Build and manage Lakehouse architectures following the Medallion pattern (Bronze, Silver, Gold)
• Implement and maintain Delta Lake tables, including schema evolution, time travel, and data versioning
• Orchestrate data workflows using Databricks Workflows (Jobs) and Delta Live Tables / Lakeflow pipelines
• Implement data governance, access controls, and lineage using Unity Catalog
• Integrate Databricks with AWS services such as S3, Glue, IAM, Lambda, Kinesis, and Secrets Manager
• Develop real-time and near real-time data processing solutions using Spark Structured Streaming
• Monitor, troubleshoot, and optimize Spark jobs and cluster performance and costs
• Implement data quality checks, validation rules, and alerting mechanisms
• Apply CI/CD and infrastructure-as-code practices for Databricks and AWS resources
• Collaborate with data, analytics, and engineering teams to deliver data-driven solutions

Technical Requirements

• 3+ years of hands-on experience with Databricks
• 2+ years of experience working with AWS cloud services
• Strong experience with Apache Spark, PySpark, and Spark SQL
• Solid experience with Delta Lake and Lakehouse architecture principles
• Experience with Unity Catalog for data governance, permissions, and lineage
• Experience building and orchestrating pipelines with Databricks Workflows and/or Delta Live Tables
• Proven experience in Spark performance tuning (partitioning, caching, Z-Ordering / Liquid Clustering, cluster sizing)
• Strong Python and SQL skills
• Ability to translate business requirements into scalable data solutions
• Strong communication skills in English

Nice to Have

Databricks certifications (Data Engineer Associate / Professional)

AWS certifications (Solutions Architect, Data Engineer)

Experience with Terraform or Databricks Asset Bundles

Experience with streaming technologies such as Kafka or Amazon Kinesis

Experience with MLflow or supporting ML workloads on Databricks

Experience with Databricks SQL and BI tools (Power BI, Tableau)

Contract

B2B

Location

Romania (Remote)