Data Engineer

IMPORTANT LOCATION & WORK AUTHORIZATION REQUIREMENTS

This is a hybrid position based in Duluth, Georgia. Candidates must currently reside within approximately 30 minutes of the Duluth office and be able to meet the required onsite schedule.

Employment sponsorship is not available for this position. Candidates must be authorized to work in the United States without current or future employer sponsorship.

Our client is seeking a Data Engineer to help build and modernize its data pipeline architecture. The immediate focus of this role is supporting the migration of ETL workloads from Redshift stored procedures and legacy SSIS packages into scalable, maintainable cloud-native pipelines using AWS Glue and S3.

Looking ahead, the team is evolving from traditional batch-oriented processing toward near real-time analytics using stream processing technologies such as Apache Flink and ClickHouse. You will work within a cross-functional engineering team alongside senior engineers, contributing to production data pipelines while expanding your expertise in modern cloud data architecture.

What You Will Do

• Build and maintain modern ETL/ELT pipelines using AWS Glue, S3, and related AWS services to replace legacy stored procedures and SSIS jobs.
• Develop data transformation workflows that are testable, version-controlled, maintainable, and observable.
• Support and optimize the Redshift data warehouse, including materialized views, query performance, and cost efficiency.
• Contribute to the transition from batch ETL toward near real-time stream processing.
• Assist with the evaluation and implementation of streaming technologies such as Apache Flink, ClickHouse, Kafka, Kinesis, or similar platforms.
• Design and support pipelines capable of handling both batch and near real-time workloads as the platform evolves.
• Collaborate with product and analytics teams to ensure data models support reporting, AI/ML initiatives, and customer-facing features.
• Follow and help refine established patterns and best practices for data pipeline development.
• Participate in production support, troubleshooting, and incident response for data infrastructure.

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 2–4 years of professional data engineering experience.
• Hands-on experience with AWS Glue using PySpark/Python, S3, and Redshift.
• Experience migrating or supporting the migration of ETL workloads from legacy technologies such as SSIS, stored procedures, or similar tools into modern cloud-native pipelines.
• Strong SQL skills, including SQL development best practices and linting, preferably within Redshift or another columnar/MPP database.
• Experience with or strong interest in stream processing frameworks such as Apache Flink, Spark Streaming, Kafka Streams, or similar technologies.
• Familiarity with data pipeline orchestration, monitoring, and error-handling patterns.
• Familiarity with infrastructure-as-code and CI/CD concepts as applied to data pipelines.

Preferred Qualifications

• Experience with Aurora MySQL or other relational databases.
• Experience with NoSQL technologies such as DynamoDB.
• Hands-on experience with near real-time OLAP technologies such as ClickHouse, Apache Druid, or similar platforms.
• Exposure to streaming data infrastructure such as Kinesis, Kafka, or MQTT.
• Familiarity with IoT or utility/metering data.
• Experience with dbt, Airflow, or AWS Step Functions.
• Ability to explain data architecture concepts to both technical and non-technical stakeholders.
• Working knowledge of Agile, Scrum, or similar software development methodologies.
• Strong problem-solving skills and a results-oriented approach.