Data Engineer (AI & Infrastructure)
Technology
Flexi IT1 week ago
Full time
Job description
About the Role
As a fast-growing startup building a next-generation platform for the gig economy, we are looking for a Data Engineer to enhance our end-to-end data infrastructure to support our AI capabilities. You will be involved in building the data backbone that powers our job matching algorithms and AI features, ensuring high-quality data flows seamlessly from our mobile and web applications into our analytical and AI environments.
Key Responsibilities
Data Engineering & Pipeline Architecture (70%)
- Scalable ETL/ELT: Design, build, and maintain automated pipelines to collect, clean, and transform data from our Flutter mobile app, web portal, and/or APIs into PostgreSQL, MongoDB, and/or Google BigQuery.
- Hybrid Storage Management: Manage and evolve our data environment, focusing on PostgreSQL for relational data and MongoDB for unstructured conversational data and transcripts.
- AI Data Readiness: Implement data quality checks, monitoring, and alerting to ensure that the data feeding our AI interview and SOP systems is reliable and trustworthy.
- Event Tracking: Facilitate the setup of custom event tracking and user behavior logging to capture the raw data necessary for AI model training.
- Matching Algorithms: Build and iterate on scoring models using vector embeddings and NLP techniques to improve job-worker fit and recommendation quality.
- Vector Search Infrastructure: Implement and optimize vector search capabilities (e.g., using Gemini API Embeddings or Vertex AI) to enable high-speed similarity matching.
- Collaborative ML: Work with our AI Engineer to deploy and monitor machine learning models that predict churn, satisfaction, and platform economics.
- Support data needs for product experimentation and growth initiatives.
- Document data models, pipelines, and business logic for team knowledge sharing.
- Stay up-to-date with new data tools and best practices; suggest improvements to our stack as we scale.
- Experience: 4+ years of experience in data engineering, or a closely related technical role.
- Technical Stack: Proficiency in SQL (PostgreSQL) and NoSQL (MongoDB) and hands-on experience with BigQuery.
- Pipeline Tools: Experience building processes using GCP tools like Cloud Functions, Dataflow, or Airflow.
- AI/ML Basics: Hands-on experience with vector embeddings or NLP libraries (e.g., Vertex AI, Gemini Embedding).
- Collaboration: General awareness of software development (APIs, Git) to work effectively with our Flutter and Backend developers.
- Experience with the GCP ecosystem (Vertex AI, Cloud Composer).
- Prior work in marketplace platforms, gig economy, or matching/recommendation systems.
- Exposure to data governance basics like access controls and PII handling in a startup context.
- The opportunity to shape the data strategy of a scaling startup.
- A collaborative environment where your work directly impacts product and operational efficiency.
Keywords
FlutterBusiness logicMongodbAirflowPostgresqlSqlMobile appBigQuery
Interested in this role?