Skip to main content

Lead Data Engineer – Databricks

Technology
1 week ago
Service contractOn-site

Job description

πŸš€ We're Hiring: Lead Data Engineer – Databricks

πŸ“ Location: USA – Remote

We are looking for an experienced Lead Data Engineer with strong hands-on expertise in Databricks, Advanced SQL, and AWS to join our team!

πŸ”Ή Key Skills & Requirements:

  • 5+ years of hands-on experience with Databricks .
  • Strong expertise in Medallion Architecture (Bronze, Silver & Gold) .
  • Advanced SQL, PySpark, and performance tuning.
  • Pipeline optimization and DBU cost optimization across All-Purpose and Job Clusters.
  • Data ingestion from multiple source systems.
  • Parsing and processing JSON/XML data.
  • Ingesting data from APIs into Delta Tables.
  • Hands-on experience with Databricks Auto Loader .
  • Publishing data to AWS services such as S3, SQS, and SNS .
  • Experience working with relational and non-relational databases.
  • Strong proficiency in Unity Catalog .
πŸ”Ή Good to Have:
  • Declarative Pipelines.
  • DBUtils commands.
  • Knowledge of broader AWS services.
🎯 Looking for candidates with strong hands-on Databricks experience and expertise in building, optimizing, and managing scalable data engineering solutions.
Keywords
azure-databricksdatabrickssqlamazon-web-servicesmedallion-architecturepysparkvehicle-modification-tuningexpense-optimizationjsonxml-formatloadersnosqlunity-3dunity-and-unreal-software-and-trainingdata-engineering

Interested in this role?

Similar jobs