Sr Data Engineer
Send a job offer directly to this candidate
With a decade of experience in the IT field, I have honed my skills as a versatile data engineer, transitioning from a role as a data warehouse developer/ETL developer to focusing on data engineering in the past five years, currently serving as an Azure Data Engineer at American Express.
In my roles, I have demonstrated expertise in designing and optimizing end-to-end ETL pipelines, employing technologies such as Azure Data Factory, Snowflake modeling, Kubernetes, and fault-tolerant design. My proficiency extends to ELT/ETL pipeline development using Python, Spark-SQL, and various Azure services. I am adept at collaborating with DevOps for CI/CD frameworks, scripting with Python and Scala, and utilizing Hive, Spark, Kafka, and Spark Streaming for real-time data processing.
With a decade of experience in the IT field, I have honed my skills as a versatile data engineer, transitioning from a role as a data warehouse developer/ETL developer to focusing on data engineering in the past five years, currently serving as an Azure Data Engineer at American Express.
In my roles, I have demonstrated expertise in designing and optimizing end-to-end ETL pipelines, employing technologies such as Azure Data Factory, Snowflake modeling, Kubernetes, and fault-tolerant design. My proficiency extends to ELT/ETL pipeline development using Python, Spark-SQL, and various Azure services. I am adept at collaborating with DevOps for CI/CD frameworks, scripting with Python and Scala, and utilizing Hive, Spark, Kafka, and Spark Streaming for real-time data processing.
Additionally, I have showcased my dedication as a data engineer by ensuring high-quality reference data through AWS services like EMR, S3, DynamoDB, and Redshift. My skills include leveraging PySpark and Python for data transformation, creating ETL processes with AWS Glue, and orchestrating workflows in Databricks. I am well-versed in Apache Spark, Hive, Hadoop, and have demonstrated proficiency in managing the end-to-end data lifecycle.
Masters in computer and Information Systems
