Big Data Developer

Big Data Developer

Toronto, ON, Canada

Contract

Job Overview

• Optimize enterprise data platforms with Spark Hadoop.
• Build and performance-tune distributed data processing applications, consolidating multi-source data into unified Hadoop data lakes while ensuring production-grade reliability.

Data Processing Development

• Build/deploy Spark/Scala applications for distributed data processing.
• Consolidate source systems into Hadoop-based data lakes.

Performance & Reliability

• Optimize Spark jobs and troubleshoot Hadoop cluster issues.
• Monitor production workflows and resolve pipeline failures.

Engineering Excellence

• Implement data modeling and ETL best practices.
• Leverage Git CI/CD for version-controlled deployments.

Must-Have Technical Skills

• Hadoop ecosystem mastery (HDFS, YARN, Hive).
• Spark Scala proficiency for large-scale processing.
• Git/CI-CD pipeline experience.
• ETL principles and data modeling.