Rushik Bisen
Data Engineer | PySpark | SQL | Python | AWS | Azure | Snowflake | Databricks | Airflow
Hire this person
Send a job offer directly to this candidate
About
Data Engineer with 4+ years of experience designing, developing, and optimizing scalable ETL/ELT pipelines and cloud-native data platforms. Skilled in Python, SQL, PySpark, Apache Spark, Databricks, Azure Data Factory, Snowflake, and DBT, with hands-on expertise in data ingestion, transformation, Delta Lake, Medallion Architecture, and workflow orchestration across Azure and AWS. Experienced in processing structured and semi-structured data, implementing data quality, governance, and security best practices, and delivering analytics-ready datasets for BI and advanced analytics.
Proven ability to build secure, high-performance data engineering solutions using Azure Synapse Analytics, ADLS Gen2, AWS S3, AWS Glue, Redshift, Apache Airflow, Docker, Git, CI/CD, and Agile methodologies.
Experience
Data Engineer at CGI (2023-08 – Present)
Healthcare Data Integration Platform for a Leading Global Pharmaceutical & Clinical Research Organization. Designed and developed an enterprise-scale clinical data integration platform on AWS to ingest, transform, and manage clinical trial data from CTMS, EDC, SQL Server, and laboratory systems. Built scalable ETL pipelines using AWS Glue and PySpark, implemented Medallion Architecture, orchestrated workflows with Apache Airflow, and enabled Snowflake analytics through Snowpipe.
- Designed and developed scalable ETL pipelines using AWS Glue and PySpark.
- Implemented full-load and watermark-based incremental loading.
- Built Medallion Architecture on Amazon S3 using partitioned Parquet.
- Developed Apache Airflow DAGs for orchestration and recovery.
- Automated Snowpipe ingestion into Snowflake.
- Performed data quality validation and business rule implementation.
- Optimized Spark jobs using partitioning, Broadcast Joins, caching, and predicate pushdown.
- Monitored production pipelines using CloudWatch and resolved incidents within SLA.
- Implemented AWS IAM and AWS KMS security controls.
- Collaborated with cross-functional Agile teams.
- Processed 30–50 GB of clinical data daily.
- Delivered analytics-ready datasets supporting 2,500+ business users.
- Improved ETL performance through Spark optimization.
- Ensured FDA and GxP compliant data governance.
Data Engineer at CGI (2023-08 – Present)
Customer 360 Analytics Platform for a Global Research & Advisory Organization. Developed an enterprise Customer 360 Analytics Platform on Microsoft Azure using ADF, Databricks, Delta Lake, and ADLS Gen2 to deliver curated datasets for retention analytics and Power BI reporting.
- Developed metadata-driven ETL pipelines using Azure Data Factory.
- Built Medallion Architecture with ADLS Gen2 and Delta Lake.
- Developed PySpark notebooks for cleansing, MDM mapping, and validation.
- Implemented Delta MERGE-based incremental loading.
- Created Databricks Workflows for orchestration.
- Optimized Spark using AQE, Broadcast Joins, OPTIMIZE, Z-ORDER, and VACUUM.
- Published curated datasets to Azure Synapse and Azure SQL Database.
- Implemented data quality validation and production support.
- Configured Unity Catalog, RBAC, Managed Identity, and Key Vault.
- Worked in Agile using Azure DevOps, Git, and Jira.
- Built a scalable Customer 360 analytics platform.
- Delivered trusted datasets for retention and churn analytics.
- Improved Delta Lake performance using optimization techniques.
- Implemented enterprise-grade governance and secure access management.
Data Engineering Intern at Icode Technology (2022-08 – 2023-08)
Education
B.Tech – G.H. Raisoni College of Engineering (2019 – 2023)
