Divyansh Pradhan

Data Engineer with 2+ years of experience designing and operating scalable ETL/ELT pipelines, cloud data platforms, and analytics solutions using Python, PySpark, SQL, Snowflake, AWS, GCP, and Databricks. Proven track record delivering high-quality data products with Airflow-orchestrated workflows, dbt transformations, and Medallion Architecture. 7x Salesforce Certified, SnowPro Core & Advanced Data Engineer Certified, and ThoughtSpot Professional & Cloud Architect Certified.

Data Engineer at Atrium (2024-01 – Present)

• Engineered and maintained enterprise-grade data pipelines across Salesforce, Snowflake, and AWS S3, supporting large-scale analytical workloads and downstream reporting systems for cross-functional business teams.
• Designed and implemented modular ETL/ELT workflows using dbt and advanced SQL on Snowflake and Databricks, enabling schema evolution, transformation logic reuse, and end-to-end data lineage.
• Established a Snowflake-based Medallion Architecture (Bronze–Silver–Gold) integrated with AWS S3 to systematically separate raw ingestion, curated datasets, and analytics-ready data marts.
• Developed PySpark jobs on Databricks for large-scale data transformation and batch processing, optimizing performance for distributed workloads on multi-TB datasets.
• Built and orchestrated automated data pipelines using Apache Airflow (Astronomer), scheduling ETL DAGs for incremental ingestion, data quality checks, and dependency management across production workflows.
• Implemented comprehensive data quality validation frameworks including freshness checks, nullability rules, and referential integrity constraints using dbt tests, achieving 99.9% accuracy across business-critical datasets.
• Optimized analytical query performance on Snowflake and BigQuery using micro-partition pruning, clustering strategies, and query profiling, significantly reducing execution latency and warehouse compute costs.
• Architected near real-time and scheduled ingestion pipelines using Snowpipe, Streams, and Tasks for incremental data processing and event-driven workloads on AWS and GCP environments.
• Collaborated cross-functionally with analytics, product, and business teams to translate complex requirements into scalable data models and high-performance analytical solutions.
• Designed and delivered Tableau dashboards powered by dbt semantic layers, enabling scalable self-service analytics for business and leadership stakeholders.
• Maintained Git-based version control workflows, enforced CI/CD best practices, and authored technical documentation for all pipeline components to ensure maintainability and team knowledge sharing.

SQL Intern at Celebal Technologies (2023-06 – 2023-08)

• Worked with large-scale relational datasets to support analytical reporting and ad-hoc analysis across multiple business domains.
• Authored and optimized complex SQL queries involving joins, window functions, and aggregations to improve query execution efficiency.
• Improved reporting performance by 40% through query refactoring, execution plan analysis, and index optimization techniques.
• Automated data validation and reporting workflows using Python scripts, reducing manual effort and improving overall data reliability.

Bachelor of Technology in Artificial Intelligence (Computer Science) – Swami Keshvanand Institute of Technology, Management & Gramothan (2020-08 – 2024-05)

Higher Secondary Education – St. Anselm's School (2019 – 2020)