Job description
π We're Hiring: Lead Data Engineer β Databricks
π Location: USA β Remote
We are looking for an experienced Lead Data Engineer with strong hands-on expertise in Databricks, Advanced SQL, and AWS to join our team!
πΉ Key Skills & Requirements:
- 5+ years of hands-on experience with Databricks .
- Strong expertise in Medallion Architecture (Bronze, Silver & Gold) .
- Advanced SQL, PySpark, and performance tuning.
- Pipeline optimization and DBU cost optimization across All-Purpose and Job Clusters.
- Data ingestion from multiple source systems.
- Parsing and processing JSON/XML data.
- Ingesting data from APIs into Delta Tables.
- Hands-on experience with Databricks Auto Loader .
- Publishing data to AWS services such as S3, SQS, and SNS .
- Experience working with relational and non-relational databases.
- Strong proficiency in Unity Catalog .
- Declarative Pipelines.
- DBUtils commands.
- Knowledge of broader AWS services.
Keywords
azure-databricksdatabrickssqlamazon-web-servicesmedallion-architecturepysparkvehicle-modification-tuningexpense-optimizationjsonxml-formatloadersnosqlunity-3dunity-and-unreal-software-and-trainingdata-engineering
Interested in this role?
