Data Engineer - SQL, Python, Java

Senior Data Engineer Position Overview We are seeking an experienced Data Engineer to design, build, and maintain scalable, reliable data platforms that support analytics and operational needs. The ideal candidate will collaborate with data scientists, analysts, and engineering teams to ingest, transform, and deliver high-quality data for both batch and real-time use cases, while ensuring performance, governance, and observability across the data ecosystem.

Key Responsibilities

• Design, implement and maintain scalable, high-performance ETL/ELT pipelines for batch and streaming data processing.
• Build and operate real-time and streaming data solutions using Apache Kafka, Apache Flink, Apache Beam or equivalent technologies.
• Develop data processing jobs with Apache Spark (PySpark/Scala/Java) and manage resource optimization and tuning for large-scale workloads.
• Design and maintain data warehouses and lakes (Snowflake, BigQuery, Redshift, Azure Synapse, Databricks) including schema design (star schema, dimensional modeling) and partitioning strategies.
• Implement robust data ingestion from a variety of sources into cloud storage (S3, Azure Blob, Google Cloud Storage) and data lake formats (Parquet, Avro, Delta Lake, ORC).
• Author and maintain data orchestration workflows using Airflow, DBT or similar tools; schedule, monitor and alert on pipeline health.
• Ensure data quality, metadata management and lineage tracking; collaborate on data governance, schema evolution and documentation.
• Optimize storage and query performance through indexing, columnar storage techniques and performance tuning (Presto/Trino, Hive, Cassandra, columnar warehouses).
• Containerize and deploy data services using Docker and Kubernetes; manage infrastructure-as-code with Terraform and maintain CI/CD pipelines with Git-based workflows.
• Implement monitoring and logging for data platforms (Prometheus, Grafana, ELK or similar) and respond to incidents, perform root cause analysis and remediate production issues.
• Collaborate with cross-functional teams to translate business requirements into technical designs and reusable data products.
• Mentor junior engineers, promote best practices for coding, testing, and operational excellence across the data organization.

Qualifications

• Bachelors or Masters degree in Computer Science, Engineering, or related field, or equivalent practical experience.
• 5+ years of hands-on experience building production data platforms and pipelines in cloud or hybrid environments.
• Strong programming skills in SQL and at least one of: Python, Java, or Scala; experience with PySpark and Spark job development.
• Proven experience with streaming systems and frameworks such as Apache Kafka, Kafka Streams/KSQL, Apache Flink or Apache Beam for low-latency processing.
• Deep knowledge of Hadoop ecosystem components (HDFS, Hive) and modern data lake architectures and formats (Parquet, Avro, Delta Lake, ORC).
• Experience with cloud data warehouses and analytics platforms (Snowflake, Google BigQuery, Azure Synapse, Amazon Redshift, Databricks).
• Familiarity with NoSQL and relational databases (Cassandra, MongoDB, MySQL, PostgreSQL, Oracle) and query engines (Presto/Trino).
• Experience designing ETL/ELT processes, data modeling (star schema, dimensional modeling) and data warehousing best practices.
• Solid experience with workflow orchestration and transform tools such as Airflow and DBT; knowledge of Apache NiFi is a plus.
• Practical experience with Linux/Bash, containerization (Docker), orchestration (Kubernetes) and infrastructure-as-code (Terraform).
• Strong Git workflows and CI/CD experience for data pipelines and services.
• Knowledge of metadata management, data lineage, schema evolution, JSON handling and data governance practices.
• Experience implementing monitoring, logging and alerting for data infrastructure (Prometheus, Grafana, ELK), plus performance tuning and partitioning strategies.
• Excellent problem-solving and communication skills, ability to work collaboratively across teams and mentor others.
• Comfortable working with CSV and other common file formats, and experience handling large-scale data transfer and storage (S3, Azure Blob, Google Cloud Storage).

In this position, you may have access to client or customer systems, confidential and/or proprietary information or data. This position is onsite and requires you to work closely with other individuals in a collaborative team environment.

Benefits

Creative Circle's Freelance Employee benefits package includes eligibility for Minimum Essential Coverage (MEC) medical plan, dental/vision/term life package, discount prescription program, critical illness, accident, tele-behavioral health, and 401(k) plan. Sick leave is provided to Candidates whose assignment work location is in a state or city subject to sick leave laws. A Minimum Value (MV) PPO medical plan, Employee Stock Purchase Plan, and paid holiday eligibility are based on length and dates of service.

Email Your Resume In Word To Rajani.Devarakonda@jobalert.creativecircle.com

Looking forward to receiving your resume through our website and going over the position with you. Clicking apply is the best way to apply.

Please do NOT change the email subject line in any way. You must keep the JobID: linkedin : VJ3-1997588L696 -- in the email subject line for your application to be considered.

Rajani Devarakonda - Genpact For Creative Circle to represent you for this opportunity, you must be currently authorized to work in the United States without the need of employer sponsorship for a non-immigrant visa such as a H-1B, TN, or O visa. We do not support or provide training for STEM/OPT programs. Additionally, you must be physically located in and perform the work for our client in the United States.This is a new role.

This job was first posted by Creative Circle on 09/20/2026 and applications will be accepted on an ongoing basis until the position is filled or closed.

Everforth Creative Circle is an Equal Opportunity Employer All qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex (including pregnancy, childbirth, breastfeeding, or related medical conditions), age, sexual orientation, gender identity or expression, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, disability, status as a crime victim, protected veteran status, or any other characteristic protected by law. Our hiring process includes AI screening for keywords and minimum qualifications.

Recruiters review all results.

Everforth Creative

Circle will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable state and local law, including but not limited to the Los Angeles County Fair Chance Ordinance, the San Francisco Fair Chance Ordinance, and the California Fair Chance Act. If you need a reasonable accommodation in the application process, please contact your Recruiter (the person you'll be interviewing with) or a member of our Human Resources team to make arrangements. Anthem / Blue Cross Blue Shield creates and publishes the Transparency in Coverage Machine-Readable Files on behalf of Everforth Creative Circle.

Copyright © 2026 Everforth, Inc. All rights reserved.