Python Developer

Greetings!

We are seeking a senior Python / PySpark Developer to join a high-impact data engineering initiative for a leading global financial institution in Whippany, NJ. In this role, you will design, engineer, and optimize distributed data pipelines and analytics engines supporting risk calculations, regulatory compliance reporting, transaction processing, and enterprise data lake modernization.

Job Title: Python / PySpark Developer

Domain: Banking & Financial Services

Location: Whippany, NJ (Hybrid: 3 Days Onsite, 2 Days Remote)

Contract Duration: 12+ Months (with potential extension)

Work Authorization: Open to all authorized candidates (W2 only)

Interview Process: In-Person Interview onsite at the Whippany, NJ campus.

Key Responsibilities

• Design, develop, and maintain high-throughput batch and real-time distributed data pipelines using Python and PySpark.
• Ingest, transform, and clean multi-terabyte financial datasets across structured, semi-structured, and unstructured sources.
• Optimize PySpark jobs by analyzing DAG execution plans, tuning memory management, addressing data skew, and refining partition strategies.
• Implement robust data quality checks, reconciliation routines, and audit logging to ensure strict adherence to banking data governance standards.
• Collaborate with upstream transaction systems and downstream consumption layers (BI dashboards, risk analytics models, and regulatory platforms).
• Build automated CI/CD deployment pipelines, automated testing frameworks, and monitoring alerts for production data jobs.
• Partner with engineering leads in Agile standups, code reviews, and architectural design sessions during onsite collaboration days.

Required Technical Skills & Qualifications

• Professional Experience: 6 to 9+ years of overall software and data engineering experience, with a solid footprint in Banking, Capital Markets, or Financial Services.
• Python & PySpark: Advanced proficiency in Python (OOP, collections, multiprocessing) and hands-on expertise building scalable pipelines with PySpark (DataFrames, Datasets, Spark SQL).
• Performance Tuning: Proven ability to debug and optimize Spark applications (broadcast joins, caching, shuffle partition tuning, Catalyst optimizer analysis).
• Database & Querying: Strong SQL writing skills with experience working with relational databases (PostgreSQL, Oracle) as well as modern analytical data warehouses (Snowflake, Databricks Delta Lake, or Redshift).
• Cloud & Orchestration: Practical familiarity running workloads in public cloud environments (AWS/Azure) utilizing services like S3, EMR/Glue, and workflow orchestrators (Apache Airflow, Step Functions).
• Financial Domain Knowledge: Understanding of banking data domains such as Trade Lifecycle, PnL, Market/Credit Risk, Fixed Income, or Regulatory Reporting (e.g., CCAR, Basel).

Best Regards,