Thoo Kok Loon

Passionate and creative Senior Software Engineer with over 8 years of experience architecting and building large-scale distributed systems, cloud-native platforms, and AI-powered applications. Proven expertise designing production-grade agentic AI systems, Retrieval-Augmented Generation (RAG) platforms, and LLM-powered workflows using LangGraph, LangChain, OpenAI, and AWS Bedrock. Strong background in Python and Golang development, building highly scalable microservices, event-driven architectures, and fault-tolerant backend systems deployed on AWS and Kubernetes.

Experienced in leading the end-to-end development of intelligent applications that automate complex cognitive workflows, including information retrieval, document intelligence, semantic search, question answering, summarization, and decision-support systems. Demonstrated success designing observable, resilient, and maintainable distributed systems supporting enterprise-scale workloads, with deep expertise in cloud infrastructure, containerization, CI/CD, and operational excellence. Passionate about advancing AI-driven products through scalable engineering practices, technical leadership, and cross-functional collaboration.

Proven ability to mentor engineers, drive architectural decisions, and translate emerging machine learning and agentic AI capabilities into impactful production solutions that improve business outcomes and user experiences.

Kayrros | Sep 2024 – Apr 2026

Role: Software Engineer – Back-end and AI Engineer

Location: Houston, TX

Responsibilities:

· Architected and built a scalable RAG-based platform capable of processing and retrieving billions of multi-format documents, including PDFs, HTML, and images, to power intelligent search and downstream AI/LLM applications.

· Led development of a React and TypeScript platform that enabled business users to search, analyze, and interact with AI-generated insights across large-scale document repositories. Built reusable React components, custom hooks, and shared UI libraries, while developing responsive dashboards and data visualization interfaces for document analytics, AI workflow monitoring, and operational reporting.

· Implemented advanced search experiences with filtering, sorting, pagination, and real-time updates, integrating React applications with FastAPI backend services and AWS-hosted APIs while enforcing secure authentication and role-based access controls.

· Designed and implemented a high-throughput, event-driven ingestion pipeline using asynchronous processing patterns to efficiently handle large-scale document workloads. Integrated the Unstructured framework for document parsing, layout-aware chunking, and metadata extraction, significantly improving semantic quality and retrieval effectiveness.

· Designed autonomous AI workflows using LangGraph to orchestrate document retrieval, embedding generation, vector indexing, tool execution, and LLM-powered analysis across distributed AWS services. Developed agentic AI services that combined semantic search, multi-step reasoning, and specialized tool invocation to automate knowledge discovery and document intelligence workflows.

· Built a modular embedding platform using registry and adapter design patterns to support multiple embedding providers and foundation models through AWS Bedrock, including Titan, Cohere, Claude Sonnet, and Nova, enabling flexible model selection based on performance, quality, and cost considerations.

· Generated high-quality vector embeddings and indexed them into Amazon OpenSearch using k-NN vector search. Designed embedding pipelines supporting both large-scale batch processing and incremental updates for continuously arriving data streams.

· Developed high-performance semantic search APIs using FastAPI, implementing Max Marginal Relevance (MMR), HNSW vector search optimization, Redis-backed caching, and concurrency tuning to improve retrieval quality and reduce p99 latency by 69% under sustained traffic spikes.

· Designed a fully decoupled event-driven architecture leveraging Amazon S3, SNS, and SQS to coordinate distributed processing workflows. Implemented retry mechanisms, dead-letter queue handling, and fault-tolerant processing strategies to ensure reliability and resilience at scale.

· Stored and managed processed document data in Amazon S3 and utilized Docker-based local environments to run OpenSearch clusters for development, testing, data backfilling, and production dataset validation through snapshot and restore workflows.

· Refactored backend and infrastructure architecture using Python-based AWS CDK, eliminating circular dependencies and enforcing clear one-directional dependency flows between application services, AWS Lambda functions, and infrastructure components, resulting in improved maintainability and service boundaries.

· Designed and provisioned cloud infrastructure using AWS CDK, deployed containerized services on Amazon ECS with Docker, and established CI/CD pipelines using Jenkins and Slack integrations to automate deployments and provide real-time operational visibility across development and production environments.

· Implemented comprehensive observability using OpenTelemetry, Splunk, and AWS CloudWatch, while contributing to system architecture, design reviews, and production readiness initiatives across OpenSearch, PostgreSQL, ECS, networking, messaging, and storage services.

· Reduced AI operating costs by 15% through prompt optimization, response caching, and model selection strategies while maintaining response quality, system reliability, and user experience.

 

Technologies: React, TypeScript, Python, FastAPI, LangGraph, LangChain, AWS Bedrock, OpenSearch, PostgreSQL, Redis, AWS S3, SNS, SQS, ECS, Lambda, AWS CDK, Docker, Jenkins, OpenTelemetry, CloudWatch, Splunk, RBAC, IAM.

Texas Medical Center | Apr 2020 – Aug 2024

Role: Software Engineer – AI Back-end, Data Science

Location: Houston, TX

Responsibilities:

· Architected and implemented an end-to-end document ingestion, processing, and indexing pipeline for large-scale legal and healthcare corpora (PDF, HTML) using Python and Apache Airflow, incorporating spaCy-based semantic segmentation, metadata enrichment, and chunking strategies optimized for downstream retrieval quality.

· Engineered high-throughput embedding pipelines using Sentence Transformers (all-MiniLM-L6-v2), generating large-scale dense vector representations and indexing them in Milvus with optimized ANN structures (HNSW, IVF) to support highly efficient, low-latency similarity search over millions of documents.

· Designed and productionized dense retrieval workflows by encoding queries with shared embedding models, executing top-k nearest neighbor search in FAISS, and applying cross-encoder reranking (BERT-based) to significantly improve precision, relevance, and answer grounding.

· Built AI-driven workflow orchestration services that combined retrieval, reranking, prompt generation, and LLM inference into automated multi-step decision-support pipelines.

· Implemented agentic retrieval workflows capable of dynamically selecting knowledge sources, retrieving relevant context, and generating grounded responses for healthcare and legal document analysis.

· Built a clinician-facing AI copilot leveraging RAG, semantic retrieval, and LLM-based summarization to generate contextual patient visit briefs from longitudinal healthcare records, enabling faster clinical review and evidence-grounded decision support.

· Engineered a patient-support AI assistant using agentic retrieval workflows and LLM tool orchestration, enabling users to access healthcare information, navigate care resources, and receive context-aware guidance through secure, retrieval-grounded interactions.

· Engineered advanced RAG pipelines using LangChain, dynamically constructing retrieval-augmented prompts with strict token budget enforcement, context window optimization, and prompt templating to ensure stable and cost-efficient LLM inference.

· Developed and deployed low-latency RAG-based Q&A microservices using FastAPI, encapsulating retrieval, reranking, prompt construction, and generation into scalable APIs capable of handling concurrent enterprise workloads.

· Integrated Hugging Face transformer models (BERT, ELECTRA, and generative LLMs) into the retrieval and generation stack to deliver context-aware, compliance-aligned responses for sensitive healthcare and legal use cases.

· Automated continuous knowledge base updates using AWS Lambda and S3 event-driven pipelines, enabling seamless ingestion, re-embedding, and FAISS index refresh with zero/minimal downtime in production environments.

· Designed GPU-accelerated inference pipelines leveraging Docker, Kubernetes, and TensorRT, optimizing batch inference, model loading, and memory utilization to significantly reduce embedding and generation latency.

· Built distributed data processing and embedding workflows using Apache Airflow, AWS Batch, and parallel Python workers, reducing end-to-end ingestion and indexing time for large datasets by parallelizing compute-intensive stages.

· Provisioned and managed scalable, reproducible cloud infrastructure using Terraform and AWS services (ECS, EKS, Lambda), enabling fault-tolerant deployments, environment consistency, and safe model/version rollouts.

· Developed robust evaluation frameworks for retrieval and generation quality using Recall@K, BLEU, ROUGE, and custom domain-specific metrics, incorporating human-in-the-loop validation to ensure high accuracy for regulated outputs.

· Strengthened system security and governance by implementing OAuth2-based authentication, encrypted data storage, and fine-grained access controls, ensuring compliance with enterprise and healthcare data protection standards.

· Built a real-time observability and evaluation pipeline for the RAG system using Apache Kafka for event streaming and Apache Flink for stateful stream processing, enabling continuous monitoring of embedding drift, retrieval quality, latency distributions, and response consistency through integrated logging, metrics, and tracing.

Technologies: React, TypeScript, Python, FastAPI, LangChain, Hugging Face Transformers, BERT, spaCy, Milvus, FAISS, Apache Airflow, Kafka, Flink, AWS Batch, Lambda, ECS, EKS, S3, Docker, Kubernetes, Terraform, OAuth2, OpenTelemetry.

 

Stirista | Dec 2017 - Mar 2020

Role: Software Engineer – Back-end, Full-stack

Location: San Antonio, TX

Responsibilities:

· Architected and delivered low-latency, high-throughput Go microservices (Gin, Echo, go-kit, gRPC) powering RTB(real-time bidding) and serving pipelines, consistently meeting sub-50ms p99 latency targets under high-QPS traffic.

· Owned end-to-end bidder/serving-path services, including OpenRTB bid request parsing, eligibility filtering, dynamic pacing algorithms, frequency capping, budget enforcement, and auction decisioning across distributed systems.

· Engineered high-throughput Kafka streaming pipelines (Go producers/consumers) processing billions of daily events (impressions, clicks, conversions) to support real-time analytics, attribution modeling, and downstream ML-driven decisioning.

· Implemented advanced Kafka consumer group architectures with partition-aware scaling, manual offset management, ordering guarantees (per key), replay/backfill pipelines, and consumer lag monitoring for deterministic and fault-tolerant processing.

· Hardened event processing reliability using idempotent handlers, deduplication keys (event IDs), exponential retry/backoff strategies, DLQ pipelines, and circuit breaker patterns to safely handle duplicates, poison messages, and downstream failures.

· Optimized streaming performance via micro-batching, compression (Snappy/LZ4), async processing pipelines, backpressure controls, and fine-grained concurrency tuning, reducing consumer lag and stabilizing throughput during peak traffic bursts.

· Designed and implemented production-grade REST and gRPC APIs with strict schema validation, rate limiting, caching layers, middleware orchestration, and strongly typed service contracts for internal service-to-service communication.

· Built low-latency data access layers using PostgreSQL and Redis for hot-path reads/writes, applying advanced indexing strategies, query optimization, connection pooling, and cache-aside patterns to consistently meet strict p95/p99 SLAs.

· Enhanced observability across distributed services using structured logging (Zap/Logrus), Prometheus metrics, Grafana dashboards, and distributed tracing (OpenTelemetry), significantly improving incident detection and root-cause analysis.

· Architected event-driven microservice ecosystems using Kafka and Redis Streams to decouple ingestion, processing, and serving layers, enabling scalable asynchronous workflows and fault isolation.

· Developed real-time fraud detection and screening services (IP/domain reputation scoring, GeoIP validation, anomaly signals) integrated directly into the bidding path with ultra-low-latency Go APIs.

· Built and maintained internal bidder CRM and operations platform using PHP (Symfony 6) and Google Cloud Bigtable, enabling campaign configuration, bidder tuning, operational workflows, and full audit traceability.

· Containerized Go services using Docker and deployed on Kubernetes, configuring liveness/readiness probes, resource requests/limits, HPA autoscaling, and zero-downtime rolling deployments to meet high availability targets.

· Improved Kubernetes runtime efficiency by tuning pod-level concurrency, connection reuse, and graceful shutdown handling, reducing tail latency and minimizing request drops during deployments and rescheduling events.

· Architected and optimized bidder services within a Demand-Side Platform (DSP), handling high-volume bid requests, applying real-time decisioning such as targeting, frequency capping, and budget pacing, and executing dynamic bidding strategies under strict SLA constraints in high-throughput production environments.

· Engineered CI/CD pipelines for DSP and low-latency bidder services using GitHub Actions and Docker, integrating automated testing, high-throughput load simulations replicating real-time bidding traffic, and progressive delivery strategies (blue/green and canary releases) to guarantee safe, zero-downtime deployments in latency-critical, high-QPS production environments.

Technologies: Go, Gin, Echo, go-kit, gRPC, REST APIs, OpenRTB, Kafka, Redis, Redis Streams, PostgreSQL, Google Cloud Bigtable, Docker, Kubernetes, GitHub Actions, Bitbucket Pipelines, Prometheus, Grafana, OpenTelemetry, Zap, Logrus, DSP, RTB, Microservices, Event-Driven Architecture, High-Concurrency Systems.

Bachelors of Computer Science: BS | 2011 - 2015

Rice University, Houston – Houston, TX

Master's Degree of Computer Science (Scientific Computing & Computer Science) | 2015 - 2017

Rice University, Houston – Houston, TX