Akash Rai

AI Engineer with 2+ years of hands-on experience building secure, focusing on both enterprise and local-first GenAI systems, optimizing LLM inference on NVIDIA and AMD hardware, and deploying production document-intelligence, RAG, agentic, and knowledge-graph platforms. Experienced in vLLM-based inference, Blackwell GPU optimization, backend and microservice architecture, air-gapped deployments, and end-to-end client delivery.

AI Engineer at Inferigence Quotient (2025-02 – Present)

• Architected and deployed an end-to-end document intelligence platform for the Indian Navy, evolving RAG-based report generation into a secure, on-premise local-LLM platform for document analysis, generation, operational suggestions, and decision support.
• Designed and deployed an air-gapped microservices platform with 5+ Dockerized services, RabbitMQ, MongoDB, MinIO, OCR pipelines, and vLLM inference; led multiple multi-day on-site client deployments in Mumbai, collaborating with stakeholders on requirements, production feedback, upgrades, and rollout in secure environments.
• Independently developed an agentic geospatial assistant supporting MBTiles-based map visualization, temporal reasoning, playback, and Vega-Lite-powered conversational analytics for interactive operational decision support.
• Built a long-term memory knowledge base that continuously ingests scanned documents through OCR, extracts timelines and operational entities, preserves supporting evidence, and revises previously stored facts when newer documents supersede earlier information.
• Optimized Ministral-3-8B FP8 on NVIDIA RTX 5060 Ti (Blackwell) using FP8 KV cache and prompt-aware batching, extending context length from 22K to 40K while serving batches of 10–20 requests at ∼750 output tokens/s aggregate throughput using approximately 15 GB VRAM.
• Enabled Gemma-4-12B NVFP4 inference on SM120 Blackwell hardware by resolving CUDA 12.9, FlashInfer, CUTLASS, JIT-cache, and kernel-build compatibility issues; containerized Qwen3-9B, Ministral-3-8B, and Gemma deployments using vLLM and Docker Compose.
• Designed an independent GPU-accelerated gRPC inference service by refactoring statistical and Random Forest classifiers into a CuPy-powered microservice, accelerating large-scale batch predictions while avoiding PyTorch/CUDA dependency bloat across downstream services.
• Conducted R&D on adapting local SLMs through continued pre-training (DAPT) and PEFT/LoRA; curated military-domain corpora from books and operational documents, implemented deduplication and preprocessing pipelines, trained models on 4× NVIDIA L40S GPUs using AWS multi-GPU infrastructure, and benchmarked domain-specific performance against base-model baselines to evaluate deployment readiness.

Independent AI Engineer – Consulting at LeanAgile Nautics / Upwork (2025-10 – 2026-05)

• Owned the end-to-end architecture and development of an enterprise analytics assistant, replacing an inaccurate RAG-first design with LLM-generated SAQL over Salesforce Einstein Analytics to answer metric-heavy and numerical business questions correctly.
• Built routing, query generation, validation, execution, result formatting, fallback logic, document ingestion, and FastAPI endpoints for production use across enterprise logistics workflows.
• Delivered a second grounded knowledge system combining Neo4j graph retrieval, vector embeddings, and semantic search to answer questions across long finance, product, and marketing documents with evidence-backed context.

AI Developer Intern at AI Planet (2024-09 – 2025-02)

• Built a multi-agent assessment generator using LangGraph state machines, orchestrating LLM agents through a 7-stage workflow for job-aligned question generation and adaptive candidate evaluation.
• Developed modular components for GenAI Stack, a drag-and-drop LLM pipeline builder, while experimenting with agent architectures and domain-specific workflow design.

AI Intern at DeepLogicAI (2024-03 – 2024-08)

• Developed a production-grade medical RAG pipeline with FastAPI, replacing Vectara-based RAGaaS to reduce operational costs by 10x while improving control over retrieval and deployment.
• Engineered a cascading retrieval pipeline combining dense retrieval, filtering, reranking, and context selection to balance recall, precision, and computational cost.
• Migrated large-scale document metadata from Pinecone to MySQL and designed an optimized indexing schema for faster filtering and improved query workflows.

Bachelor of Engineering in Electronics and Communication – NMIT Bengaluru (2021-12 – 2025-07)