Software Engineer, DevOps (WFH)

# Principal Software Engineer, DevOps

## About the Role

Legion is seeking a Principal Software Engineer, DevOps to build and operate a secure, highly scalable, and cost-effective AWS and Kubernetes-based cloud platform. As part of the Site Reliability Engineering (SRE) team, you will work across infrastructure tooling, automation, CI/CD, monitoring, logging, containerization, and cloud services that keep Legion’s platform reliable for customers.

The role also focuses on applying AI to SRE operations. You will build and operate internal AI agents that reduce engineering toil through incident triage, runbook execution, alert enrichment, capacity and cost analysis, and other operational workflows. You will also own the platform that enables engineers to safely build and operate agents with appropriate sandboxing, scoped credentials, tool and MCP integrations, cost controls, guardrails, rollback capabilities, and observability.

## Key Responsibilities

• Support Legion’s public cloud platform using multiple cloud services and containerization technologies.
• Develop infrastructure automation using Terraform, Chef, Jenkins, and Golang.
• Create and manage production alerts.
• Respond to production alerts and conduct root-cause investigations.
• Develop automated operational runbooks.
• Support deployment of Legion’s Workforce Management (WFM) solution during and outside regular office hours.
• Participate in an on-call rotation.
• Build and operate internal AI agents to reduce SRE toil, including:
• Incident triage
• Runbook execution
• Alert enrichment
• Capacity analysis
• Cost analysis
• Own guardrails, rollback mechanisms, and reliability for internal AI agents.
• Build and operate the platform that enables engineers to safely develop AI agents.
• Implement sandboxed agent execution, scoped credentials, tool and MCP integrations, cost controls, and agent observability.
• Introduce AI into engineering workflows and the software development lifecycle where appropriate.
• Evaluate where AI should and should not be applied within engineering processes.
• Support and scale observability systems and cloud infrastructure.
• Collaborate effectively with remote engineering teams.

## Required Qualifications

• 12+ years of experience in SRE, DevOps, or other SaaS operations.
• 3+ years of hands-on experience managing AWS services and cloud infrastructure using Terraform.
• 3+ years of experience with containerized cloud solutions using Docker, Kubernetes, or AWS EKS.
• Familiarity with Helm charts.
• 5+ years of experience with at least one programming language such as Go, Python, Bash, or Perl.
• 5+ years of experience with one or more of the following:
• Infrastructure automation tools such as Terraform or Ansible
• CI/CD pipelines and tools such as Git and Jenkins
• Configuration management tools such as Ansible or Chef
• Hands-on experience with Linux/Unix platforms such as Red Hat, CentOS, Ubuntu, or Amazon Linux.
• Hands-on experience building production agentic AI systems, including tool integrations, evaluation, and failure handling, beyond the use of AI coding assistants.
• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related field.

## Preferred Qualifications

• AWS Certification.
• 3+ years of experience with observability tools and platforms such as:
• Splunk
• Nagios
• Elasticsearch
• Kibana
• CloudWatch
• Logstash
• Experience scaling observability systems.
• 3+ years of experience with AWS RDS or Aurora MySQL.
• Demonstrated track record of introducing AI into an engineering organization’s SDLC with measurable results.
• Strong judgment regarding appropriate and inappropriate use of AI within engineering workflows.
• Demonstrated ability to work effectively with remote teams.

## Technical Skills

• Site Reliability Engineering (SRE)
• DevOps
• AWS
• Cloud Infrastructure
• Kubernetes
• Docker
• AWS EKS
• Terraform
• Chef
• Jenkins
• Golang / Go
• Python
• Bash
• Perl
• Linux / Unix
• Helm
• CI/CD
• Infrastructure Automation
• Configuration Management
• Monitoring and Alerting
• Root-Cause Analysis
• Operational Runbooks
• Observability
• Splunk
• Nagios
• Elasticsearch
• Kibana
• CloudWatch
• Logstash
• AWS RDS
• Aurora MySQL
• Agentic AI
• AI Agents
• MCP Integrations
• AI Evaluation
• Agent Guardrails
• Sandboxed Execution
• Agent Observability
• Reliability Engineering

## Work Arrangement & Schedule

• Location: Remote, United States
• Work Arrangement: Fully remote
• Participation in an on-call rotation is required.
• Support for Legion’s WFM solution may be required during and outside regular office hours.
• The role involves working with remote teams.

## Compensation

• Base Salary Range: $220,000–$255,000
• Stock equity included as part of the compensation package.
• Compensation is determined using local, national, and industry-specific survey data.
• Final salary may be refined based on the candidate’s region, town, or cost of living.
• Compensation considers location, experience, education, specialized knowledge and skills, external market rates, internal budget, and internal equity.

## Benefits

• $0 monthly premium medical coverage and other flexible medical, dental, and vision plans effective from the first day of employment.
• 401(k) plan.
• Discretionary Paid Time Off.
• Paid holidays.
• Parental leave.
• Equity.
• Monthly wellness reimbursement.
• Monthly lunch at Legion.