Job description
Job Description:
The Data Engineer works within the ETL and Operations team to build high quality data pipelines driving analytic solutions from diverse, disparate sources of data.
This role requires a comprehensive understanding of data architecture, data engineering, and data analysis. The ideal candidate is a skilled data engineer with experience creating data products supporting analytic solutions. They are able to identify and implement solutions in a highly technical environment and work as part of a technical, cross-functional team. Strong problem-solving and troubleshooting skills are a must.
This role focuses on designing, developing, and maintaining datasets within AWS. Day-to-day responsibilities include:
- Coordinating, building, and managing new data ingests
- Architecting updates, fixes, and optimizations across suite of production jobs
- Providing feedback on and enacting changes for improvements across the team including both technical and process updates
- Learn and assess new technologies for implementation through proof-of-concept projects and testing
- Mentoring junior developers
- Partnering with data analysts and data scientists to design, build, and deploy aggregation processes
- Hadoop/Hive experience
- Very Strong SQL
- AWS experience (EMR, Lambda, Glue, Step Functions)
- Experience working with large data sets (billions of records per day)
- Write and review code
- Python scripting (PySpark, control scripts)
- Architecting/designing data pipelines
- Query and data pipeline optimization
- Strong communication and collaboration skills
- Ability to help others and mentor junior resources
- Spark
- Scala
- Git
- Shell scripting
- Linux
Required Skills : - Experience manipulating data with SQL (MUST BE VERY STRONG IN ADVANCED SQL FUNCTIONS) - AWS: Athena, S3, EMR, Lambda - nice to have - Python or PySpark - nice to have
Basic Qualification :
Additional Skills :
Background Check :Yes
Drug Screen :Yes
Notes :Hybrid role (50% remote) / 50% onsite in any of the following markets - (St. Louis, MO; Charlotte, NC; Dallas, TX; Denver, CO; Madison, WI; Stamford, CT)
Selling points for candidate :
Project Verification Info :yes will provide
Exclusive to Apex :No
Face to face interview required :No
Candidate must be local :No
Candidate must be authorized to work without sponsorship ::No
Interview times set : :No
Type of project :Development/Engineering
Master Job Title :Big Data: Dev
Branch Code :Denver
Interested in this role?
