Full Stack Python/Data Engineer

Job Description:

Top 6 Technical Skills:

• Python (Big Data Pipeline)
• AWS
• Hadoop, Spark, Hive
• EMR
• Terraform
• Databricks

Job Description:

• Location is Reston (hybrid model 3x a week in office)
• Strong Python development to build a big-data pipeline for data processing and analysis
• Need strong experience in Hadoop EMR, Spark, Pyspark, Hive, AWS, Terraform (end to end automation), Github, CICD
• Need to work within a team of developers to assist with development
• Need to be open to collaborating and advising on best possible solutions and tools to use

Nice to have:

• AI (Some AWS AI background is nice to have, manager didn''t remember which tools they use exactly)
• Vulnerability management background (Tenable, Wiz, Kenna)
• Financial / Mortgage based background

Qualifications

• 10 years hands-on Python development experience for big data application.
• Extensive working experience in implementing scalable and efficient data processing pipelines using big data technologies, such as Hadoop/EMR, Spark, Hive.
• Strong development experience in AWS platforms/services such as Lambda, Eventbridge, Step Functions, Redshift, S3, Glue.
• Extensive hands-on development experience in making API calls, SQL queries, data manipulation/transformation, analysis and visualization.
• Strong experience in data analysis/mining, hands-on experience on analyzing large datasets and extracting meaningful insights.
• Solid understanding and hand-on implementation experience in CI/CD pipeline. Familiar with tools such as Gitlab/Terraform.
• Critical thinking and strong analytical skills to drive solutions, alternative plans, and conclusions
• Collaborates effectively and forms positive partnerships with team members in groups and levels across a matrixed organization
• Possess strong analytical and problem-solving skills

Skills

• Required 10 years of Python development experience for big data application
• Required 5 years of development experience in AWS platforms/services such as EMR, Lambda, Eventbridge, Step Functions, Redshift, S3, Glue
• Required 5 years of development experience in making API calls, SQL queries, Spark, data manipulation/transformation, analysis and visualization
• Hands on experience with CI/CD pipeline tools such as Gitlab/Terraform