Job Title
Python/PySpark Engineer
Skill
Digital : Python
Minimum Experience
10 - 15 Years
Qualification
BACHELOR OF COMPUTER SCIENCE
Location
Jersey City, NJ
Job Description
Must Have Technical/Functional Skills
Technical Skills
- Strong proficiency in Python, packaging, dependency management, and virtual environments.
- Hands-on experience with PySpark, including Spark performance tuning (partitioning, caching, broadcast joins, memory optimization).
- Expertise in data ingestion (batch/stream), schema management, and robust error-handling/retry logic.
- Solid unit and integration testing practices, including data quality validations.
- Experience with CI/CD pipelines (Azure DevOps/Jenkins), Git branching strategies, and artifact versioning.
- Working experience with Cloudera/Hadoop (HDFS, Spark, Hive/Impala) and Databricks (Delta Lake, clusters, jobs, notebooks).
- Knowledge of observability techniques: structured logging, metrics, tracing, and debugging in distributed systems.
- Secure coding practices including secrets management, PII protection, and compliance-aware development.
Functional Skills
- Strong documentation discipline for frameworks, reusable components, and best-practice patterns.
- Effective collaboration with Cloud Architects and Data Ops to ensure stable and supportable pipelines.
- Clear communication of technical ideas and solution approaches.
- Comfort working in Agile environments with iterative development and frequent releases.
Roles & Responsibilities
- Design and develop scalable Python/PySpark ingestion and transformation pipelines.
- Implement schema evolution logic, validation frameworks, and resilient error-handling mechanisms.
- Optimize Spark jobs for performance, cost efficiency, and production readiness.
- Integrate all jobs into automated CI/CD pipelines, ensuring versioning and release governance.
- Work closely with Ops teams to ensure proper monitoring, logging, and operational supportability.
- Participate in Agile ceremonies, sprint planning, code reviews, and demo sessions
TCS Employee Benefits Summary:
Discretionary Annual Incentive.
Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
Family Support: Maternal & Parental Leaves.
Insurance Options: Auto & Home Insurance, Identity Theft Protection.
Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
Time Off: Vacation, Time Off, Sick Leave & Holidays.
Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
KV#
