Shivam Hota

Integrated data from SAP ECC, SAP ATTP, SAP GTS, SAP MDG, Kinaxis and SAP BW into Kafka and Amazon S3, enabling a Unified Data Hub (UDH). Designed and implemented data transformations using Data Flows for both batch and delta processing, handling and enriching 50+ GB of data daily using timestamp-based incremental logic. Built and optimized data transformation logic for cleansing, standardization, and enrichment, ensuring high data quality and consistency across large datasets.

Data Engineer - Accenture - Bengaluru, IN

(2024-05)

• Integrated data from SAP ECC, SAP ATTP, SAP GTS, SAP MDG, Kinaxis and SAP BW into Kafka and Amazon S3, enabling a Unified Data Hub (UDH).
• Designed and implemented data transformations using Data Flows for both batch and delta processing, handling and enriching 50+ GB of data daily using timestamp-based incremental logic.
• Built and optimized data transformation logic for cleansing, standardization, and enrichment, ensuring high data quality and consistency across large datasets.
• Created custom Python Operators for audit SQL generation, workflow automation, and real-time data enrichment.
• Used Task Chains to orchestrate multiple Data Flows, managing dependencies between ingestion and transformation steps, and ensuring end-to-end data synchronization with persistence into local tables.

Associate - Cognizant - Bengaluru, IN

(2021-07 - 2024-05)

• Led ECC and BW migration to S/4HANA and Azure Data Lake, handling 50M+ records across Finance and Analytics modules.
• Developed SAP Data Intelligence Cloud (DIC) pipelines/graphs for FI-GL objects and payment interfaces, processing 20–30 GB data daily.
• Extracted data from CDS Views using Gen1/Gen2 graphs and ABAP Operators, ensuring high-throughput transfer of millions of records with accuracy.
• Built custom Python Operators to generate audit SQLs, enrich XML data for 10K+ payment transactions per day, manage port signal flows, and automate configurations.
• Integrated data from Azure Data Lake to SAP See4C by implementing API-based pipelines using Python requests module; retrieved data via GET requests from Azure and sent enriched data to See4C using POST requests with payload, enabling audit logging of millions of records monthly for compliance and monitoring.
• Contributed to both development and monitoring, ensuring end-to-end stability, data quality, and reliability of pipelines.
• Facilitated mandatory Python upgrade from 3.9 to 3.11 for 50+ pipelines, updating dependencies, libraries, and Dockerfile configurations to ensure compatibility and stability.

B.Tech. - Electrical and Electronics Engineering - VIT, Vellore (2017 - 2021)