Iris Software - Senior Data Engineer - Python/PySpark

Job Title : Data Engineer

Experience : 6- 9 Years

Summary :

We are seeking an experienced Data Engineer with 6- 9 years of expertise in designing, developing, and optimizing large-scale data platforms and pipelines. The ideal candidate should have strong hands-on experience in Python, PySpark, Snowflake, ETL development, and Big Data Architecture, with the ability to build scalable and high-performance data solutions that support analytics and business intelligence initiatives.

Roles &

Responsibilities :

• Design, develop, and maintain scalable data pipelines for batch and real-time data processing.
• Architect and implement robust Big Data solutions to support enterprise-wide analytics and reporting needs.
• Build and optimize ETL/ELT workflows using Python, PySpark, and modern data engineering frameworks.
• Develop and manage data models, data warehouses, and data marts within Snowflake environments.
• Collaborate with data analysts, data scientists, and business stakeholders to understand and translate data requirements into technical solutions.
• Ensure data quality, integrity, governance, and security across multiple data sources and platforms.
• Optimize data processing jobs, query performance, and storage utilization to improve efficiency and reduce costs.
• Monitor, troubleshoot, and resolve issues related to data ingestion, transformation, and pipeline performance.
• Implement best practices for CI/CD, code reviews, testing, documentation, and deployment of data engineering solutions.
• Stay current with emerging technologies and recommend improvements to enhance the organization's data architecture and engineering capabilities.

Requirements :

• 6- 9 years of experience in Data Engineering, Data Warehousing, or Big Data technologies.
• Strong programming skills in Python with experience in developing scalable data processing applications.
• Hands-on expertise in PySpark for distributed data processing and transformation.
• Proven experience designing and implementing Big Data architectures and data lake solutions.
• Strong experience with Snowflake, including data modeling, performance tuning, and warehouse optimization.
• Extensive experience building and maintaining ETL/ELT pipelines for large-scale data environments.
• Good understanding of data warehousing concepts, dimensional modeling, and database design principles.
• Experience working with cloud platforms such as AWS, Azure, or GCP is preferred.
• Familiarity with orchestration and workflow tools such as Airflow, Azure Data Factory, or similar platforms.
• Strong analytical, problem-solving, communication, and stakeholder management skills with the ability to work in an Agile environment.

Preferred Qualifications :

• Experience with streaming technologies such as Kafka or Spark Streaming.
• Knowledge of DevOps practices, CI/CD pipelines, and infrastructure automation.
• Exposure to data governance, metadata management, and data quality frameworks.
• Relevant cloud or Snowflake certifications are a plus.