Skip to main content

Data Quality Engineer - Python/ETL

Technology
Forward Eye Technologies
Pune, India5 days ago
Full time

Job description

We are seeking a skilled and experienced Data Quality Engineer (Data QE) to join our team. The ideal candidate will have a strong background in Quality Engineering, ETL and database testing, and hands-on expertise with Python, PySpark, SQL, and Databricks. You will be responsible for designing and executing tests to ensure the quality and reliability of data pipelines and analytics platforms.

Key Responsibilities :

  • Design and implement end-to-end test strategies for data pipelines and ETL processes.
  • Conduct thorough ETL and database testing to ensure data accuracy, completeness, and consistency.
  • Develop automation frameworks and scripts using Python and PySpark.
  • Perform data validation and integrity checks using advanced SQL queries.
  • Work with big data platforms such as Databricks for testing and validation.
  • Collaborate closely with data engineers, analysts, and product teams to understand data requirements.
  • Identify data quality issues, troubleshoot, and work with stakeholders to resolve them.
  • Document test cases, results, and provide regular status updates.

Mandatory Skills & Experience :

  • 6 years of experience in Quality Engineering and QE processes.
  • 23 years of hands-on experience in ETL testing and database testing.
  • 23 years of experience with Python and PySpark for data testing and automation.
  • 23 years of experience in writing complex SQL queries for data validation.
  • Experience working with Databricks or similar big data platforms.

Preferred Qualifications :

  • Experience with cloud data platforms (e.g., AWS, Azure, GCP).
  • Familiarity with CI/CD pipelines and test automation tools.
  • Knowledge of data governance and data quality frameworks.

Education : Bachelors or Masters degree in Computer Science, Information Systems, or a related field.

Role: Data / Big Data Testing Engineer, Industry Type: E-Learning / EdTech, Department: Engineering - Software & QA, Employment Type: Full Time, Permanent Role Category: Quality Assurance and Testing Education UG: B.Tech / B.E. in Any Specialization PG: Any Postgraduate Doctorate: Doctorate Not Required Key Skills Skills highlighted with ‘‘ are preferred keyskills Data Quality Engineering Data QualityETLPythonPySparkSQLData ValidationData IntegrityAzure Databricks

Keywords
Data Quality EngineeringData QualityETLPythonPySparkSQLData ValidationData IntegrityAzure DatabricksCI/CDBig data

Interested in this role?