Spark Developer (With Java)

Job Title: Spark Developer (Java)

Experience: 4 to 8 Years

Location: Any

Employment Type: Full-time

Job Summary

We are seeking a highly skilled Spark Developer with strong Java expertise to design, develop, and optimize large-scale data processing applications. The ideal candidate will have hands-on experience with Apache Spark, Core Java, distributed computing, and big data technologies. The candidate will work closely with data engineers, architects, and business stakeholders to build scalable and high-performance data solutions.

Key Responsibilities

• Design, develop, and maintain scalable data processing applications using Apache Spark and Java.
• Develop batch and near real-time data pipelines for large datasets.
• Optimize Spark jobs for performance, memory management, and resource utilization.
• Write efficient and reusable code using Java 8/11 features.
• Work with distributed data processing frameworks and big data ecosystems.
• Collaborate with cross-functional teams to gather requirements and deliver data solutions.
• Troubleshoot production issues and provide performance tuning recommendations.
• Implement best practices for coding, testing, deployment, and monitoring.
• Participate in code reviews and technical design discussions.
• Ensure data quality, reliability, and scalability of applications.

Required Skills

• Strong programming experience in Core Java (Java 8 or above).
• Hands-on experience with Apache Spark (Spark Core, Spark SQL, DataFrames, Dataset APIs).
• Good understanding of distributed computing concepts.
• Experience with HDFS, Hive, Hadoop Ecosystem.
• Strong SQL skills and experience with relational databases.
• Experience with data ingestion and transformation frameworks.
• Familiarity with Linux/Unix environments.
• Knowledge of REST APIs and microservices architecture.
• Experience with Git and CI/CD processes.

Preferred Skills

• Experience with Spark Streaming, Kafka, Scala, or PySpark.
• Exposure to cloud platforms such as AWS, Azure, or GCP.
• Experience with Airflow, Oozie, or similar workflow orchestration tools.
• Understanding of Delta Lake, Iceberg, or Data Lake architectures.
• Knowledge of containerization tools such as Docker and Kubernetes.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
• 4+ years of software development experience.
• 2+ years of hands-on experience developing applications using Apache Spark and Java.