ETL testing with python/pyspark

Job Description

Location – Chennai/Bangalore/Gurugram

Experience ( 4-8 years)

• Strong understanding of ETL processes: extraction, transformation, loading
• Experience with data validation, data quality checks, and reconciliation
• Ability to design and execute test cases for large-scale data pipelines
• Knowledge of data warehousing concepts
• Understanding of Streamsets pipelines, processors, and stages
• Hands-on knowledge of: Apache Spark (PySpark/Scala),Hive / HQL and HDFS / S3 storage concepts
• Strong SQL skills
• Basic knowledge of Python / PySpark for data validation scripts

͏