Skip to main content

Role: AI Platform DevOps Engineer

Technology
Triunity Software, Inc.
1 month ago
Service contractOn-site

Job description

Key Responsibilities

  • Build and operate observability for

    LLM gateways, model routing, MCP servers, agent runtimes, and AI platform services

    .

  • Implement end-to-end telemetry using

    OpenTelemetry, Langfuse, metrics, logs, and distributed tracing

    .

  • Monitor and analyze
  • LLM/agent latency, errors, token usage, model performance, quality, evaluation results, and production behavior

    .

  • Develop dashboards, alerts, and reporting for

    AI reliability, performance, quality, and operational health

    .

  • Build
  • AI cost and usage observability

    , including attribution by application, team, user, model, workflow, and environment across providers such as OpenAI, Anthropic, and Gemini.

  • Establish observability and evaluation standards for

    LLM and agentic applications

    , including traces, prompts, responses, tool calls, evaluations, and regression signals.

  • Support
  • AI evaluation and monitoring through golden datasets, scoring, prompt/model comparisons, and production quality checks.

  • Operate AI platform services on

    Kubernetes using Helm, Kustomize, GitOps, autoscaling, and progressive/zero-downtime deployments.

  • Implement platform security, access control, secrets management, PII protection, guardrails, and audit logging

    .

Key Skills

  • AI/LLM Observability:
  • Langfuse, OpenTelemetry, LLM tracing, AI evaluation

  • AI Platforms:
  • LLM gateways, model routing, MCP, agentic AI

  • Cloud/Platform:
  • Kubernetes, Helm, Kustomize, GitOps, APIs

  • Monitoring:
  • Prometheus, Grafana, Datadog or similar platforms

  • AI/ML:
  • OpenAI, Anthropic, Gemini, LangChain, LangGraph, CrewAI, Google ADK

  • Engineering:
  • Python/Java/Go, REST APIs, CI/CD

  • Security:
  • OAuth2/OIDC, JWT, RBAC, secrets management, PII/AI guardrails

Keywords
artificial-intelligenceai-platformdevelopment-operations-devopsobservabilitylarge-language-model-llmrouting-and-switchingmicrosoft-certificationmodel-context-protocol-mcpopentelemetrynetwork-latencyerrors-omissions-e-odigital-tokenmodel-performanceassessment-assessment-toolsreporting-and-disclosureadaptive-project-management-and-reportingenvironment-health-and-safety-hsseattributionworkflowecology-environment

Interested in this role?