Data Engineer | Expert in Python, SQL, and Spark
Send a job offer directly to this candidate
With over 2.7 years of dedicated expertise, I am a Data Engineer skilled in architecting, developing, and maintaining robust data pipelines. My proficiency extends to managing Databricks clusters and orchestrating ETL workflows using Python, SQL, and Apache Spark.
I have a strong background in CI/CD methodologies, agile project management with Jira, and Kanban, coupled with effective collaboration using Confluence. My hands-on experience encompasses diverse databases, including Postgres SQL, MySQL, and Oracle, ensuring data reliability and performance.
Passionate about harnessing the power of data to drive actionable insights, I continuously strive for excellence in delivering data-driven solutions. My track record includes Amgen Rapid Response Ingestion System, and I'm dedicated to pushing the boundaries of what data engineering can achieve.
Implemented efficient ELT processes, reducing data processing time for quick insights by cleansing and transforming raw data. Managed batch and streaming data ingestion seamlessly from AWS S3 and optimized infrastructure with automated object deletion, ensuring cost-efficient management.
GitLab, Databricks, AWS (S3, Lambda), and Spark SQL for streamlined operations. Demonstrated strong soft skills in team collaboration, problem-solving, and effective communication. Achievements include a 30% efficiency boost in design handling 15 GB of data daily, a 25% reduction in data errors in clinical data flow, a 15% decrease in data processing time through diverse ingestion patterns, and exceptional performance in TCS's National Qualifier Test, ranking among the top performers nationwide.
Master of Computer Applications - MCA
