Job description
Data Engineer
*This position can sit anywhere on the East Coast
SUMMARY:
This position is responsible for design, development and management of data workflows and pipelines in our Integrated Data and Analytics environment. The role is a blend of technical leadership, team leadership, and hands-on development.
RESPONSIBILITIES/TASKS:
- Responsible for collecting, moving, storing, and preparing data for analytics enablement.
- Participates in project planning sessions, working closely with business analysts and team members to analyze requirements and provides design recommendations for complex systems.
- Designs and develops new data pipelines for ingesting high-volume data sets.
- Develops pipelines using disciplined processes, adhering to industry standards and best practices, including complete ease with agile development methods throughout the SDLC.
- Demonstrates processes to all stakeholders.
- Shares knowledge and provide technical guidance to other team members within area of expertise.
- Takes part in reviews of work, e.g. design and code reviews.
- Works with team members to analyze and understand problems with complex systems and resolve them.
- Resolves customer complaints and respond with suggestions for improvements and enhancements.
EMPLOYMENT QUALIFICATIONS:
EDUCATION:
Bachelor's degree in a related field. Relevant combination of education and experience may be considered in lieu of degree. Continuous learning, as defined by the Company's learning philosophy, is required. Certification or progress toward certification is highly preferred and encouraged.
EXPERIENCE:
2 years' experience in application development.At least five years of experience in Azure big data services/technologies (Spark, Scala, Azure Data Factory, Azure Data Bricks, Synapse Analytics).
SKILLS/KNOWLEDGE/ABILITIES (SKA) REQUIRED:
- Deep knowledge of SQL/TSQL and Python/PySpark
- Strong understanding of Azure Big Data Services such as Azure Data Lake Storage, Azure Data Factory, Logic Apps, Azure Functions
- Knowledge of AWS cloud and AWS services such as S3 buckets, Lambda, API Gateway
- Experience with Databricks and Delta Lake
- Experience with RDBMS (Oracle 10g/11g, MS SQL), including database design, developing stored procedures and functions, and performance tuning. Working experiences with No-SQL databases.
- Strong background in traditional ETL tools (SSIS, Informatica)
- Understanding of Reporting and Data Visualization tools (PowerBI, Tableau, SSRS)
- Agile software process
- Strong coding skills (Python, R, Scala, Java, C++, C#)
- Working experiences of test-driven development framework.
- Excellent inter-personal and communication skills.
- Ability to multi-task in a fast-paced environment.
- Flexibility to adapt to changing business needs.
- Team player who is delivery-oriented and takes responsibility for the team's success.
- Enthusiastic, can-do attitude with the drive to continually learn and improve.
- Highly self-motivated, creative and able to dive deep in problem solving.
Required Skills : Python PySpark/Spark DataBricks any cloud experience - Azure, AWS, GCP, etc. any ETL experience - SSIS, Informatica, DataStage, etc.
Basic Qualification :
Additional Skills :
Background Check :Yes
Drug Screen :Yes
Notes
Selling points for candidate :
Project Verification Info
Candidate must be your W2 Employee :Yes
Exclusive to Apex :Yes
Face to face interview required :No
Candidate must be local :No
Candidate must be authorized to work without sponsorship ::No
Interview times set :Yes
Type of project :Development/Engineering
Master Job Title :Dev: .NET
Branch Code :Richmond
Interested in this role?
