Deepanshu Rai

10+ years in data engineering with expertise in AWS, Python, SQL, and modern data platforms. Strong hands-on experience with Redshift, Databricks, Airflow, Glue, Kafka, and Lakehouse architectures. Experienced in building & managing high-performing data engineering teams from the ground up. Experienced in real-time data processing, data lakes, and lakehouse implementations. Adept at stakeholder collaboration and translating business needs into robust data solutions.

Manager - Data Engineering at Bain & Co (2023-08 – Present)

60% Leadership + stakeholder management + roadmap and execution; 40% hands on data engineer

• Leading the team to execute data architecture and pipelines to support enterprise tech and GenAI usecase
• Architect and develop cloud-based data pipelines and frameworks leveraging AWS, Python, SQL, and modern cloud data warehouses to ensure efficient data flow and integration across diverse business domains.
• Working with senior executives and stakeholders across Retail, Pharma, and Energy sectors to define data strategies, assess business needs, and create scalable data solutions
• Partner cross-functionally with Data Science, Analytics, and IT teams to ensure seamless integration of data assets and support advanced analytics and AI initiative

Engineering Manager - Data at CRED (2020-01 – 2023-06)

• Leading team of data engineers to make the business successful
• Building strategy from scratch for complete data ecosystem (Data Platforms + ETL + Lakehouse + Data Marts / Semantic Layer + Data Ops + Data Governance )
• Working with stakeholders to enable them with data
• Helping team to translate product/business requirement into technical capability
• Building self serve ETL platform to make Data Ops easier
• Building and ecosystem for Data Governance using various tools
• Ensuring the design is scalable and implementation is top notch

Data Engineer at Amazon (2019-03 – 2019-10)

• Maintaining the Data Lake & Data Platform for Social Ads Team.
• Writing code in Java & Python for data ingestion module.
• Creating Data Pipelines for new Data Onboarding.
• Rearchitechured existing pipelines for better throughput.
• Creating transformation based on user requirements.
• Creating Data Models based on user requirements.
• Handling Data Quality and latency of Real Time data
• Working with teams across the globe to understand the requirement and make there life easy in terms of data

Data Engineer at PharmEasy (2018-03 – 2019-03)

• Writing JAVA & Python code for Data Platform development.
• Writing MapReduce Job for ingesting data from different event based third party systems (CleverTap, AppsFlyer, Google Analytics etc) and producing the events to Kafka.
• Creating Data Lake in AWS S3 to data available for stakeholders
• Setting up Data Warehouse & ETL from scratch.
• Developed modules - Data Pipelines, Ingestion & Transformation
• Redshift Administration for better performance
• Data Modeling & creating the DAG using Apache Airflow
• Creating Jersey REST services using Dropwizard & Hibernate.

ETL Developer at DXC Technology (2015-10 – 2018-03)

• Involved in Developing the Data stage jobs for ETL.
• Developed Teradata (BTEQ, MLOAD, FLOAD & TPump) Scripts to transform the Business data into User requirements (ETL).
• Working on Shell scripts for Process Automation
• Data Modeling & Data Mart designing for OLAP Data.
• Periodically checking the performance of the system & fix the gaps

B.Tech in Computer Science & Engineering – Lovely Professional University (2011-08 – 2015-06)