Shreyas Waghmode

Engineered and deployed enterprise-scale LLM and Vision AI inference services on NVIDIA H100 GPUs using vLLM, Triton Inference Server, TensorRT, Docker Swarm, and Kubernetes, delivering secure, scalable, and low-latency production AI workloads.

Built and maintained high-throughput production inference endpoints for multimodal models including Qwen2.5-VL and its successor variants, integrating optimized inference runtimes and GPU-backed serving infrastructure for enterprise applications.

Designed production-ready containerized AI deployment pipelines using Python, Docker, Linux, and shell scripting, automating packaging, configuration, deployment, and operational management of AI inference services across distributed GPU clusters.

Developed three production GenAI applications (HR RAG, Patient Voice Agent, and Financial SQL-RAG) serving 1,000+ users by architecting deterministic vector-less RAG pipelines using SQL execution and knowledge graph traversal, eliminating embedding bottlenecks while improving retrieval reliability.

Implemented enterprise guardrail architectures using NeMo Guardrails with prompt injection detection, hallucination mitigation, PII redaction, and policy enforcement aligned with ISO 42001 and ISO 27001 compliance requirements.

Designed a conversational memory framework incorporating short-term memory, long-term memory, and fact extraction, improving multi-turn conversational consistency by more than 35%.

Performed GPU-aware deployment and inference optimization, including model configuration, batching strategy tuning, memory utilization optimization, container orchestration, and reduction of CPU-GPU synchronization overhead in speculative decoding workflows.

Contributed performance optimizations to the vLLM ecosystem through speculative decoding improvements and collaborated across AI and infrastructure teams to deploy reliable, production-grade inference systems.

ML Engineer at Anvex AI Technologies Pvt Ltd. (2025-07 – 2026-06)

• Engineered Enterprise Agent Guardrails: Implemented robust guardrail architectures (NeMo Guardrails) ensuring AI systems align with ISO 42001 and ISO 27001 compliance standards. Effectively mitigated prompt injection, suppressed hallucinations, and filtered PII for healthcare and financial agents.
• Pioneered Vector-less RAG Architectures: Delivered 3 production AI apps (HR RAG, Patient Voice Agent, Financial SQL-RAG) serving 1k+ users. Bypassed traditional embedding bottlenecks by utilizing direct SQL-schema execution and deterministic knowledge-graph traversal.
• Advanced Memory & Orchestration: Developed a GenAI Memory Layer Framework (short/long-term memory, fact extraction) improving conversational reliability by 35%+. Deployed inference services on H100 GPUs, orchestrated securely via Docker Swarm and Kubernetes.
• SOTA Fine-Tuning & Pipeline Optimization: Designed scalable data pipelines and executed GRPO/GSPO fine-tuning using Unsloth, reducing training time by 25%.
• High-Throughput Telephony & Vision CV: Built an AI Telephony Agent System running 30+ custom agents via Kafka microservices, achieving 3× throughput. Contributed to Vision CV pipelines using TensorRT-optimized models (5–8× faster), employing mechanistic interpretability techniques to audit model reasoning.
• Infrastructure: Implemented MCP-style integrations, Temporal.io tracking, GitLab CI/CD, and developed low-latency API workflows for LLM-based generative tasks.

SDE Intern (GenAI) at Alemeno Pvt Ltd. (2024-09 – 2025-02)

• Agentic Tooling: Built a custom Model Context Protocol (MCP) client/server architecture to seamlessly integrate LLMs with external tools, proprietary APIs, and active execution environments.
• Document Intelligence: Developed enterprise Q&A and workflow automation pipelines for internal teams, heavily utilizing long-context reasoning over traditional vector chunking.
• Computer Vision: Developed classification and object detection pipelines for automated LCD defect analysis.

SDE Intern (GenAI) at Soft Nerve Technologies Pvt Ltd. (2024-08 – 2024-08)

• Custom Frameworks: Created a custom lightweight LLM Agent framework, stripping away framework bloat to improve execution latency versus standard LangChain deployments.
• Cost Optimization: Reduced inference costs significantly by fine-tuning and integrating smaller, domain-specific LLMs for complex medical data workflows.

Bachelor of Engineering in Artificial Intelligence and Data Science with Minor in Artificial Intelligence in Finance and Banking – New Horizon Institute Of Technology & Management (Mumbai University) (2021-12 – 2025-06)