Job description
Project: Marketing Data Platform (used to enable real time campaigns and data for other teams). Building new cloud data warehouse solution, that is built in SnowFlake. Consume real time data through Kafka (coming from multiple sources). Data gets uploaded to SnowFlake. Framework is built, but they are in the initial stages.
Looking for
- Data Engineer with SnowFlake, Kafka, and Data Bricks (mid to senior level)
o Can look at SnowFlake or Data Bricks o Knowledge of knowing how to connect with Kafka streams
- Python
- Azure experience (can look at other Cloud solutions, but Azure is highly preferred)
- ETL
- data security and governance guideline
Nice to have
- Good communication
Team size: Scrum team that is consulting (9 people), and they are building an internal team. Consultants will be there through end of year.
Screening topics
- Databricks OR Snowflake
- Lead Data Engineer
- Can use Azure Data Engineer if they have Azure experience
- Python PySpark Coding
Please provide a summary of the project/initiative that this candidate will be working on?
The candidate will be working closely with IT team members building a robust Marketing technology platform for enabling real time campaigns and data for marketing and Data Scientist team to make quick decisions.
Please describe the team the candidate will be working with - how many members? 10
What are the top 5-10 responsibilities for this position (please be detailed as to what the candidate is expected to do or complete on a daily basis)?
- Should have prior experience managing Data lake and data warehouse type application
- Should have prior experience managing big data platform to ingest / transform time series, batch data for real time access
- 1 to 3 years of experience with Snowflake/synapse, Databricks, Kafka, eventHub, Spark.
- Should have experience with data security and governance guideline
- Should have experience managing such application on cloud (preferably Azure) platform and also understand native cloud concept such as eventHub, ADLS, Azure data factory.
- Should have experience managing various kind of BI tools for reporting like tableau, DOMO
What does the ideal candidate background look like (ex: healthcare specific background specific industry experience etc.)?
ETL, Data warehousing, Python experience
What skills/attributes are required (please be detailed as to number of years of experience for each skill)?
- Undergraduate degree or equivalent experience
- 5 + year of IT experience
- 1 to 3 years of experience with Snowflake/synapse, Databricks, Kafka, eventHub, Spark. [required]
What skills/attributes are preferred (what will set a candidate apart)?
- 1 to 3 or more years of experience with Snowflake/synapse, Databricks, Kafka, eventHub, Spark.
Required Skills : • Data Engineer with SnowFlake, Kafka, and Data Bricks (mid to senior level) o Can look at SnowFlake or Data Bricks o Knowledge of knowing how to connect with Kafka streams • Python • Azure experience (can look at other Cloud solutions, but Azure is highly preferred) • ETL • data security and governance guideline
Basic Qualification :
Additional Skills : fully remote with a large company
Background Check :Yes
Drug Screen :Yes
Notes :This is a very urgent need. The manager has filled this role twice and needs to find a third person because the others backed out. They are VERY eager to hire.
Selling points for candidate :fully remote with a large company
Project Verification Info
Exclusive to Apex :Yes
Face to face interview required :No
Candidate must be local :No
Candidate must be authorized to work without sponsorship :Yes
Interview times set :Yes
Type of project :Development/Engineering
Master Job Title :Big Data: Dev
Branch Code :Minneapolis
Interested in this role?
