Skip to main content

Data & AI Engineer - Python/SQL

Technology
Opvia.in
4 days ago

Job description

About the job :

Data & AI Engineer - Python | SQL | Azure | Data Engineering | AI/LLM | Automation

Location : Remote, India

Working Hours : UK Business Hours

Experience : 3 - 5 Years

Joining Preference : Immediate or up to 30 days' notice

About the Opportunity :

Opvia is hiring a Data & AI Engineer to work with a UK-based client on building, maintaining and improving data infrastructure, AI-powered solutions and automated workflows. You will work across data pipelines, cloud-based data systems, API integrations and AI-driven applications. The role involves developing reliable data processing solutions, integrating multiple data sources, building intelligent automation and preparing data infrastructure for scalable AI and analytics use cases.

This is a hands-on engineering role suited to someone who enjoys solving technical problems, building practical solutions and taking ownership of projects from development through deployment.

What You Will Do :

  • Design, develop and maintain scalable data pipelines using Python and SQL.
  • Build and manage ETL/ELT workflows to extract, transform, validate and load data from multiple sources.
  • Develop Python-based applications, scripts and backend services for data processing and automation.
  • Work with Azure services, including Azure Functions, App Services, Key Vault and cloud-based data infrastructure.
  • Build and integrate REST APIs, third-party platforms and external data sources.
  • Develop AI-powered applications using Large Language Models (LLMs) for data extraction, processing, classification and intelligent automation.
  • Build AI workflows, document-processing solutions, chatbots and retrieval-augmented generation (RAG) applications.
  • Work with vector databases, embeddings and AI frameworks to develop practical AI solutions.
  • Implement data validation, quality checks, error handling, monitoring and logging to ensure pipeline reliability.
  • Optimise SQL queries, database performance and data processing workflows.
  • Work with Snowflake and modern data warehouse technologies, supporting data migration and integration.
  • Use Git and GitHub for version control, code management and deployment workflows.
  • Explore and implement AI-assisted development tools such as Claude Code, Codex and Cursor to improve engineering productivity.
  • Maintain technical documentation and ensure solutions are scalable, maintainable and secure.

What We Are Looking For :

  • Strong hands-on experience with Python and SQL.
  • Experience building and maintaining ETL/ELT pipelines and data processing workflows.
  • Good understanding of data engineering concepts, database design and data modelling.
  • Practical experience working with REST APIs, integrations and structured/unstructured data.
  • Experience with cloud platforms, preferably Microsoft Azure.
  • Understanding of Azure Functions, cloud deployments, storage services and Key Vault.
  • Hands-on experience or practical project exposure to LLMs and generative AI applications.
  • Understanding of prompt engineering, embeddings, vector databases and RAG architectures.
  • Experience building AI-powered automation, data extraction or intelligent applications.
  • Familiarity with frameworks and libraries such as LangChain, LlamaIndex, OpenAI APIs or similar.
  • Experience with Git/GitHub, code debugging, testing and deployment practices.
  • Understanding of data quality, validation, monitoring and error-handling mechanisms.
  • Ability to independently develop, troubleshoot and deliver technical solutions.
  • Good written and verbal English communication skills.

Good to Have :

  • Experience with Snowflake, Databricks or other cloud data platforms.
  • Familiarity with dbt, Apache Airflow or similar orchestration tools.
  • Experience with Azure OpenAI, OpenAI API, Anthropic Claude or other LLM providers.
  • Knowledge of machine learning fundamentals and model integration.
  • Experience with vector databases such as Pinecone, ChromaDB, Weaviate or FAISS.
  • Familiarity with Docker, CI/CD pipelines and cloud deployment practices.
  • Experience with Power BI, Looker or other data visualisation tools.
  • Exposure to agentic AI, AI agents and multi-step workflow automation.
  • Azure, AWS or relevant AI/cloud certifications.

Why Join Us? :

  • Work on real-world data engineering and AI implementation projects.
  • Build practical AI-powered applications and intelligent automation solutions.
  • Gain exposure to modern cloud infrastructure and scalable data platforms.
  • Work with experienced technical teams and contribute to end-to-end engineering projects.
  • Enjoy a remote working environment with regular collaboration across UK teams.
Keywords
OrchestrationCloud computingApache AirflowCursorAirflowPythonSqlApache Http ServerCI/CDDebugger

Interested in this role?