Python Developer

About The Company

Tata Consultancy Services (TCS) is a global leader in IT services, consulting, and business solutions. With a rich history spanning over five decades, TCS has established itself as a trusted partner for organizations seeking innovative technology solutions to drive their digital transformation. Headquartered in Mumbai, India, TCS operates in over 46 countries, serving clients across diverse industries including banking and financial services, retail, manufacturing, healthcare, and more.

The company prides itself on its commitment to excellence, sustainability, and fostering a culture of continuous learning and innovation. TCS's core values emphasize integrity, responsibility, and a customer-centric approach, making it a preferred employer for technology professionals worldwide.

About The Role

We are looking for a talented and experienced PySpark Developer to join our dynamic data engineering team at TCS. The ideal candidate will play a crucial role in designing, developing, and maintaining large-scale data processing pipelines, enabling our clients to harness the full potential of their data assets. This role requires a deep understanding of Python, PySpark, SQL, and data warehousing concepts, along with the ability to work with cloud platforms and big data ecosystems.

As a PySpark Developer, you will collaborate with cross-functional teams to deliver scalable, efficient, and reliable data solutions that support business intelligence, analytics, and decision-making processes. Your expertise will directly impact the effectiveness of data-driven strategies, ensuring high performance and adherence to best practices in data engineering.

Qualifications The ideal candidate should possess a strong educational background in Computer Science, Information Technology, or related fields, with relevant professional experience.

Key qualifications include proven hands-on experience in Python and PySpark development, expertise in Spark SQL, DataFrames, and RDDs, and a solid understanding of Hadoop ecosystem components such as Hive, HDFS, and YARN. Additionally, proficiency in SQL query optimization, data warehousing concepts, and familiarity with various data formats like Parquet, Avro, and JSON are essential.

Experience with version control tools like Git, and scheduling platforms such as Airflow or Oozie, will be advantageous. Candidates with exposure to cloud platforms like AWS, Azure, or GCP will have a distinct edge. Strong problem-solving skills, attention to detail, and the ability to work collaboratively in a fast-paced environment are critical for success in this role.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines utilizing PySpark to handle large datasets efficiently.
• Build and optimize Spark jobs to enhance performance, reliability, and scalability of data processing tasks.
• Process and transform large datasets using Spark SQL, DataFrames, and RDDs to meet business requirements.
• Develop both batch and real-time data processing solutions to support diverse analytics needs.
• Integrate data pipelines with various data platforms such as Hive, HDFS, Snowflake, and Redshift, ensuring seamless data flow.
• Collaborate closely with data engineers, data analysts, and business stakeholders to understand data requirements and deliver solutions accordingly.
• Monitor data pipelines continuously, troubleshoot issues proactively, and ensure adherence to service level agreements (SLAs).
• Follow best coding practices, maintain version control, and implement CI/CD pipelines for streamlined deployment.
• Work with Hadoop ecosystem tools and leverage cloud platforms like AWS, Azure, or GCP for scalable data solutions.

Benefits At TCS, we believe in nurturing talent and providing a comprehensive benefits package to support our employees' well-being and professional growth. Our benefits include competitive salary packages, health insurance coverage, and retirement plans. Employees have access to continuous learning opportunities through training programs, certifications, and workshops to enhance their skills and stay updated with the latest industry trends. We promote a flexible work environment that encourages work-life balance and offers options for remote working where applicable. Additionally, TCS provides a vibrant and inclusive workplace culture that values diversity, innovation, and collaboration. Employees can also benefit from various wellness programs, employee assistance services, and recognition initiatives that acknowledge outstanding contributions.

Equal Opportunity

Tata Consultancy Services is an equal opportunity employer. We are committed to fostering an inclusive environment where all employees and applicants are treated with respect and dignity. We do not discriminate based on race, color, religion, gender, sexual orientation, age, disability, or any other characteristic protected by applicable laws.

Our hiring practices are designed to promote diversity and ensure equal access to employment opportunities for all qualified candidates. We believe that a diverse workforce enhances our ability to serve our clients effectively and drives innovation within our organization.