Data Engineer

Request ID: 109482-1

Title: Data Engineer

Location: Frederick, MD (Onsite)

Duration: 6 months

Pay Range: $50 - $55/Hour on W2/C2C (All inclusive)

JOB DESCRIPTION:

We are seeking an experienced Azure Databricks Data Engineer to design, develop, and maintain scalable, high-performance data engineering solutions. The ideal candidate will have strong hands-on expertise in Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, Lakehouse Architecture, and real-time data processing. The role will focus on building enterprise-grade batch and streaming data pipelines, implementing modern Lakehouse and Medallion Architecture, developing reusable ETL/ELT frameworks, and delivering trusted, governed datasets for reporting, analytics, and downstream applications.

Key Responsibilities:

• Design, develop, and maintain scalable, high-performance data engineering solutions on Azure Databricks.
• Build and support batch and real-time data pipelines using Apache Spark and Structured Streaming.
• Design and implement Lakehouse and Medallion Architecture (Bronze, Silver, Gold) using Delta Lake.
• Develop reusable ETL/ELT frameworks, standardized data processing components, and enterprise data ingestion patterns.
• Ingest and process large-scale data from diverse enterprise sources, including databases, APIs, message queues, and file-based systems.
• Develop complex data transformation and processing logic using PySpark, Python, and advanced SQL.
• Create trusted, governed, and consumption-ready datasets for reporting, analytics, and downstream applications.
• Implement data quality, validation, reconciliation, audit, and governance controls across data pipelines.
• Manage and optimize Databricks Workflows, Delta Live Tables (DLT), Unity Catalog, and Databricks SQL.
• Apply data engineering best practices for pipeline optimization, performance tuning, deployment, and operational support.
• Collaborate with business, analytics, architecture, and delivery teams in an Agile environment to deliver high-quality data products.

Required Technical Skills:

Databricks & Data Engineering

• Strong hands-on experience with Databricks, Lakehouse Architecture, and Delta Lake.
• Expert-level proficiency in PySpark, Python, and advanced SQL.
• Strong experience with Apache Spark and Structured Streaming.
• Experience with Delta Live Tables (DLT), Databricks Workflows, Unity Catalog, and Databricks SQL.
• Experience designing and implementing Medallion Architecture and modern Data Lake/Data Warehouse solutions.
• Strong background in data ingestion, ETL/ELT, data transformation, performance optimization, and deployment practices.
• Experience with event-driven and streaming technologies such as Apache Kafka or Azure Event Hubs.

Cloud Platforms

• Strong Azure experience is preferred.
• Experience with AWS or GCP is an added advantage.

Application Integration

• Working knowledge of React and/or Angular for front-end integration and data-centric application development.

Domain Experience – Preferred

• Experience in Finance, Insurance, Regulatory Reporting, or US GAAP reporting.
• Understanding of financial data models, reporting processes, and data governance requirements is highly desirable.

Company Benefits & Culture

• Opportunity to work with a dynamic team in a fast-paced environment
• Exposure to cutting-edge technologies and methodologies
• Supportive and collaborative work culture

Appreciate your quick response and please feel free to reach me out for any query you may have.