Big Data Engineer at Wipro Ltd. (2022-05 – Present)
Client: U.S. Bank | Banking & Financial Services Current Assignment
- Develop PySpark and Spark SQL pipelines to ingest, transform and process data from multiple source systems.
- Work on metadata-driven onboarding frameworks to standardize data migration for new data sources.
- Build and maintain Hive-based consumer layer solutions for downstream reporting and analytics.
- Monitor and support Apache Airflow DAGs and troubleshoot pipeline failures in production.
- Tune Spark workloads using partitioning, join optimization, caching and resource allocation.
- Work with data architects, QA and business stakeholders in an Agile delivery model.
Big Data Engineer at Wipro Ltd. (2022-05 – Present)
Client: U.S. Bank | Banking & Financial Services Conformed Layer Data Migration
- Developed PySpark ETL/ELT pipelines to migrate data from legacy systems into the Conformed Layer.
- Used Spark SQL and Hive for distributed data processing and data transformation.
- Created and optimized Hive tables and Parquet datasets using partitioning and file-compaction techniques.
- Worked on Spark SQL, Hive and Hadoop performance issues including shuffle, data skew and resource utilization.
- Used Azure Synapse Analytics for analytical processing and recurring migration activities.
- Performed source-to-target validation and reconciliation before migration sign-off.
Big Data Engineer at Wipro Ltd. (2022-05 – Present)
Client: U.S. Bank | Banking & Financial Services EAACL Consumer Layer
- Migrated legacy batch workloads to Spark-based processing.
- Developed PySpark transformations and Spark SQL logic for reporting requirements.
- Built and maintained Azure Data Factory pipelines for data movement and transformation.
- Managed AutoSys schedules, job dependencies and workflow monitoring.
- Supported production issue triage and root-cause analysis for Consumer Layer pipelines.
