Tony Zhao

Senior Software Engineer with 7+ years of software engineering experience specializing in AI/ML infrastructure, generative AI applications, backend systems, and production machine learning across enterprise data, healthcare technology, and consumer platforms. Owns complex engineering initiatives from architecture through deployment, building scalable services and intelligent workflows while providing technical direction and mentoring engineers. Experienced in Python, asynchronous APIs, LLM orchestration, retrieval-augmented generation (RAG), model evaluation, and distributed data pipelines.

Combines strong software engineering fundamentals with pragmatic AI system design to deliver reliable automation, measurable operational improvements, and decision-support capabilities.

Senior Software Engineer, AI & ML - Snowflake - Menlo Park, CA | Full-time | Remote

(2024-05 - 2026-10)

Within the Cortex AI team, contributed to Cortex AI Services, developing intelligent data and machine learning capabilities that help enterprise customers build and operate AI-powered applications. Owned backend architecture and production AI integrations, strengthening agent execution, model evaluation, and governed access to enterprise data.

• Architected Python backend services for multi-step AI workflows, using FastAPI, asynchronous execution, and modular service interfaces to support concurrent requests, tool orchestration, and reliable integration with enterprise data systems.
• Developed LLM-powered agent workflows integrating prompt templates, retrieval, structured outputs, and tool/function calling, enabling agents to execute multi-stage data analysis tasks while maintaining predictable interfaces and controlled execution paths.
• Implemented a reusable agent orchestration framework supporting conversation state, persistent memory, execution checkpoints, and streaming responses, allowing independent workflow components to coordinate without tightly coupled service dependencies.
• Designed RAG pipelines combining document ingestion, chunking, embedding generation, and semantic retrieval with structured enterprise data, improving contextual grounding and enabling more accurate responses to complex business questions.
• Built AI evaluation harnesses with curated test datasets, automated scoring, regression checks, and human feedback collection to measure response relevance, retrieval quality, tool-selection accuracy, and behavioral consistency across model changes.
• Integrated deterministic Python services and analytical models into AI workflows, separating numerical computation and validation from probabilistic LLM reasoning to improve reproducibility and prevent unsupported model-generated results from reaching downstream systems.
• Established production observability and monitoring for AI services using structured logging, distributed tracing, latency tracking, and failure analysis; collaborated with platform and product engineers on architecture reviews and mentored engineers on backend reliability and production AI development.

Senior Software Engineer, ML Infrastructure - TruEra - Redwood City, CA | Full-time | Onsite

(2022-02 - 2024-05)

As part of the AI Quality team, contributed to TruLens and AI Observability, building infrastructure that helps machine learning practitioners evaluate, explain, and monitor production AI applications. Led backend and data infrastructure improvements supporting scalable evaluation, model diagnostics, and continuous quality monitoring.

• Designed scalable Python backend services for machine learning evaluation and monitoring, separating ingestion, computation, and reporting responsibilities to support independent scaling and maintainable service boundaries.
• Developed distributed data ingestion pipelines processing model predictions, inference metadata, and evaluation feedback, implementing schema validation and normalization to produce consistent datasets for downstream diagnostics.
• Built reusable LLM evaluation and feedback mechanisms for retrieval-augmented applications, incorporating context relevance, groundedness, and answer relevance measurements to identify quality regressions across model and prompt configurations.
• Implemented model monitoring and drift detection pipelines using statistical comparisons and configurable evaluation thresholds, helping engineering and data science teams identify anomalous model behavior and investigate underlying data-quality issues.
• Optimized high-volume analytical queries and PostgreSQL data access patterns through indexing, query restructuring, and efficient batch processing, improving the responsiveness of model diagnostics and evaluation workflows.
• Introduced comprehensive unit, integration, and regression testing for evaluation services and data pipelines, including deterministic fixtures and repeatable test datasets to validate complex ML workflows across releases.
• Partnered with ML researchers, product engineers, and customer-facing technical teams to translate model quality requirements into reusable platform capabilities; mentored engineers through design reviews and established implementation patterns for reliable ML infrastructure.

Software Engineer, Data - Ejenta - San Francisco, CA | Full-time | Onsite

(2020-02 - 2022-01)

Contributed to the Intelligent Systems team and its Connected Care Platform, developing data services that transform patient monitoring and sensor information into actionable insights for healthcare providers. Built data processing and backend components supporting predictive analytics, automated monitoring, and reliable communication between intelligent agents and care teams.

• Developed Python data ingestion pipelines integrating heterogeneous sensor streams and external healthcare records, implementing normalization, schema validation, and transformation logic to provide consistent inputs for downstream analytics.
• Built backend services for processing time-series patient monitoring data, introducing asynchronous processing and configurable validation rules to support reliable handling of incoming observations and abnormality detection workflows.
• Implemented data-quality checks and anomaly detection preprocessing for predictive analytics pipelines, collaborating with machine learning engineers to improve input consistency and the reliability of downstream risk assessment.
• Designed reusable API integrations connecting monitoring services with internal applications, improving interoperability between data collection, automated notifications, and clinical workflow components.
• Established automated tests for ingestion and transformation logic, covering malformed records, missing sensor observations, and boundary conditions to improve pipeline reliability and simplify production troubleshooting.

Machine Learning Engineer - Spotify - San Francisco, CA | Full-time | Onsite

(2021-05 - 2021-08)

Worked with the Personalization team on Home Recommendations, contributing machine learning data and model-serving workflows that help listeners discover relevant music and podcasts. Focused on feature processing, model evaluation, and reliable integration of recommendation models into production personalization systems.

• Implemented feature engineering and data preprocessing workflows for recommendation models, transforming behavioral and interaction data into standardized features suitable for candidate generation and ranking.
• Developed model evaluation utilities using offline recommendation metrics and segmented analysis, helping identify ranking-quality differences and validate model changes before production deployment.
• Collaborated with machine learning engineers and data scientists to improve training and inference data consistency, strengthening validation and monitoring practices across recommendation workflows.

Software Development Engineer - Amazon - Irvine, CA | Full-time | Onsite

(2019-05 - 2020-08)

Contributed to the ML Platform team and its Model Workflow Services, building backend and data processing components that support scalable machine learning development and deployment. Focused on dependable service interfaces, automated data workflows, and testing practices for distributed software systems.

• Developed Java and Python backend services supporting internal data processing and model workflow integrations, emphasizing modular APIs, clear service contracts, and reliable handling of asynchronous operations.
• Implemented automated data validation and transformation components for machine learning workflows, improving consistency of structured inputs and reducing failures caused by malformed or incomplete records.
• Expanded unit and integration test coverage for backend services, collaborating with engineers through code reviews and debugging distributed application issues to improve release confidence and maintainability.

Master of Science - Electrical Engineering and Computer Science (EECS) - University of California, Berkeley (2020 - 2021)

Bachelor of Science - Electrical Engineering and Computer Science (EECS) - University of California, Berkeley (2016 - 2020)