Nausheen Fatima

Senior Data Engineer at ELI LILLY (2023-06 – Present)

Built a Databricks pipeline processing ~50GB/day of campaign manager 360 data via autoloader.
Designed and migrated ETL batch jobs on Databricks using PySpark supporting large scale analytics that resulted in efficient query performance and reduced runtime by ~50%.
Built a scalable data lake architecture using Apache Iceberg with AWS Glue Catalog, enabling schema evolution and exposing datasets via Redshift Spectrum for downstream analytics while streamlining compliance processes.
Automated data loading process for files with dynamic schema structures, streamlining data ingestion workflow eliminating manual effort by ~28 hours per month.
Tuned Redshift query performance through distribution key selection, sort key design, and query plan analysis.
Implemented an ETL pipeline on AWS glue to execute pre-trained ML models stored on S3, extracting tags for creatives and loading data into Redshift for downstream analytics.
Developed optimized stored procedures and SQL transformations to support large-scale analytical queries and reporting workloads.

Data Engineer at IQVIA (2021-09 – 2023-05)

Designed and built end to end data warehouse solution along with architecting the required infrastructure in AWS using VPC, Redshift, S3, enabling ingestion and analytics.
Automated process to copy files from SFTP to S3 and loading data from S3 to Redshift. Files moved to the archive folder post the completion of data processing. Integrated lambda notification with MS Teams using webhook.
Created Power BI reports using data from Amazon RDS and performed transformations and DAX to refine data and visualize the same.
Extracted data from Amazon Redshift and done transformation, data visualization. Created solution driven dashboards by developing different graphs in Tableau Desktop.
Developed Python code for data stewardship process in Master Data Management which reduced the effort by 75%.
Built Python scripts to scrape physician data from different websites and perform fuzzy matching using Alteryx.
Implemented API-based integration between Veeva CRM and data warehouse.
Orchestrated ingestion workflows using Apache Airflow DAGs, enabling scalable and reliable scheduling.
Collaborated with stakeholders through solution design discussions, client calls, and knowledge transfer sessions.

Business Technology Solutions Associate at ZS Associates (2020-07 – 2021-09)

Worked as part of the development team to create and automate the data pipelines, providing reporting solutions, designing the ETL frameworks and data modeling.
Oversees the data load production process and implementation of new data load files.
Developed multiple python scripts to automate the data process like - handling file modifications before getting loaded into the system.
Developed a python script that moves files from portal to S3, improving ingestion reliability.

About