Devin Brown

Staff Software Engineer with 11+ years of experience architecting scalable software platforms, distributed systems, cloud-native applications, and full-stack products. Experienced in technical leadership, Generative AI, large language models (LLMs), Retrieval-Augmented Generation (RAG), recommendation systems, CI/CD modernization, and production observability. Skilled in Python, FastAPI, Azure OpenAI, LangChain, Azure AI Search, TypeScript, React, and Ruby on Rails.

Demonstrated impact in reducing infrastructure costs, accelerating software delivery, strengthening test coverage, and improving the scalability and maintainability of production systems.

Staff AI/ML Software Engineer - Sana Benefits - Austin, Texas

(2021-06)

• Led technical initiatives supporting virtual care, contributing to scalable healthcare services and stronger application architecture.
• Designed an Enterprise Knowledge Assistant powered by Azure OpenAI and GPT, enabling natural-language access to enterprise documentation and knowledge sources.
• Architected a custom Retrieval-Augmented Generation (RAG) pipeline with LangChain to retrieve relevant enterprise content and supply contextual evidence for LLM-generated responses.
• Built document-ingestion workflows spanning parsing, chunking, metadata enrichment, embedding generation, indexing, and retrieval to prepare enterprise content for AI-powered search.
• Integrated Azure AI Search to support semantic retrieval across enterprise knowledge sources.
• Developed Python services and FastAPI REST endpoints to connect the AI assistant with enterprise applications and knowledge repositories.
• Applied prompt engineering, context construction, grounding, and response-validation techniques to improve the relevance and reliability of generated answers.
• Structured the solution into modular document-processing, retrieval, LLM-orchestration, and integration services; containerized workloads with Docker and deployed them using Kubernetes on Azure.
• Improved large-scale provider-directory ingestion pipelines and led a rebuild of provider-care search tooling, reducing system footprint and simplifying the underlying architecture.
• Strengthened operational visibility through observability improvements and led a Technical Quality Group focused on scale-critical KPIs.
• Collaborated with engineering leadership to decouple and modularize a legacy Ruby on Rails, C# monolith, supporting a more maintainable application architecture.

Senior Software Engineer(Early-Stage Technology Incubator ) - Geometer LLC - Denver, Colorado

(2020-10 - 2021-05)

• Explored high-concurrency application architectures within an early-stage technology incubator focused on the Erlang Virtual Machine (BEAM).
• Investigated architectural approaches for scale-intensive, real-time systems, including voice/video communications and massively multiplayer applications.

Senior AI/ML Software Engineer - CircleCI - San Francisco, California

(2019-06 - 2020-09)

• Built and fine-tuned NLP and semantic-retrieval models with TensorFlow and Hugging Face Transformers, using BERT and RoBERTa encoders; applied tokenization, text normalization, hard-negative mining, in-batch negatives, embeddings, semantic similarity, and task-specific fine-tuning for search and recommendation.
• Developed retrieval and text-understanding evaluation workflows using Recall@K, MRR, nDCG, precision, calibration, slice analysis, error taxonomies, explainability, and regression datasets to detect quality changes before launch.
• Designed candidate generation with approximate-nearest-neighbor indexes, including ScaNN, followed by learned reranking across semantic, lexical, freshness, and behavioral signals; trained LightGBM and neural ranking baselines and selected models through reproducible evaluation.
• Built BigQuery and Dataflow pipelines for labels, hard negatives, feature generation, entity resolution, and point-in-time training datasets, with attention to feature leakage and reproducibility.
• Built Python/FastAPI model-administration services, a Node.js/TypeScript backend-for-frontend layer, and a React/Next.js experimentation workspace for query review, candidate inspection, ranking comparison, feedback analysis, and controlled launch decisions.
• Integrated model outputs through FastAPI, Node.js/GraphQL, React/Next.js, Redis, PostgreSQL/BigQuery, and gRPC, exposing explanations, confidence, feedback, and launch-control data to product and operations teams; kept evaluation and ranking logic in Python services.
• Exposed retrieval and ranking through versioned gRPC and Protocol Buffers APIs with latency budgets, caching, tracing, schema compatibility, and fallbacks; implemented request batching, Redis caching, pagination, rate limiting, structured error handling, and typed API clients.
• Optimized model serving through request batching, quantization experiments, ONNX conversion, resource profiling, and autoscaling while maintaining evaluation parity; built monitoring for feature freshness, retrieval quality, ranking distributions, latency, errors, saturation, and business outcomes.
• Designed controlled A/B tests, guardrail metrics, hypothesis tests, and post-launch analysis to distinguish model lift from traffic changes; implemented privacy-aware feature access, service identity, encryption, audit logging, retention controls, and security-review evidence.
• Extended retrieval experiments to image/text embeddings and multimodal content understanding and asset similarity, with offline evaluation, human review, and vector-search indexing alongside text retrieval.
• Coached engineers on ranking design, feature leakage, calibration, experimentation, code quality, production debugging, and technical decision records.

Senior Software Engineer - Pivotal - San Francisco, California

(2015-07 - 2019-05)

• Led a ground-up redesign of the CI/CD platform, reducing IaaS operating costs by 85% and cutting build and deployment times from approximately 50 minutes to under 10 minutes.
• Increased automated test coverage from approximately 40% to more than 90%, improving regression protection and confidence in software changes.
• Built workflow automation, CI/CD status pages, and notification systems to improve visibility into delivery pipelines and reduce manual coordination.
• Designed a Content Discovery and Recommendation Service using user interests, content characteristics, and behavioral signals to surface relevant content.
• Developed content-processing workflows, feature-engineering pipelines, and recommendation and ranking logic to support personalized discovery.
• Integrated recommendation capabilities into backend services and REST APIs, connecting data pipelines with application-facing functionality.
• Modernized frontend applications from Backbone and jQuery toward ES6, React, and Redux, while optimizing browser rendering and network performance.
• Built and supported real-time event-propagation systems for a multi-tenant application handling 20,000+ requests per second and 100,000+ concurrent connections.
• Developed distributed data aggregation, logging, visualization, and alerting capabilities to support troubleshooting across complex systems.
• Created an SVG/canvas visualization library backed by D3 to support interactive data visualization.
• Redesigned application observability, helped establish internal SRE service-level objectives, and contributed to Pivotal Tracker’s migration to cloud-based hosting.

Full Stack Engineer - Splick.it - Boulder, Colorado

(2014-08 - 2015-07)

• Built interactive frontend features using JavaScript, ReactJS, AngularJS, and Redux for a food-ordering and delivery platform.
• Designed RESTful APIs using Ruby on Rails, Python, Node.js, .Net and Express.js to connect application interfaces with backend services.
• Partnered with UX/UI designers to translate wireframes and mockups into functional, user-focused interfaces.
• Integrated third-party services, including payment gateways and document-signing platforms, into application workflows.
• Improved application performance and scalability through caching, database-query optimization, and load-balancing strategies.

Associate Software Engineer - MapQuest - Denver, Colorado

(2013-05 - 2014-08)

• Contributed to the development and maintenance of mapping software and geospatial application functionality.

Bachelor of Science (BS) - Computer Science and Mathematics - Metropolitan State University of Denver (2008-01 - 2012-05)