Przejdź do treści głównej

Linux Kernel Live Patching Engineer

Technology
Sardina Systems
4 dni temu
Pełny etatStacjonarna

Opis stanowiska

Sardina Systems is an OpenInfra software platform company based on OpenStack and Kubernetes and is looking for a Packager/Package Maintainer to join our team.

Linux Kernel Live Patching Engineer

Build and operate a live kernel patching feed for AlmaLinux 10. You own the full technical chain, from CVE triage to signed, tested RPMs published in a public repository. This is a hands-on kernel engineering role, not a management or release-coordination position.

Core responsibilities

Patch engineering

  • Track AlmaLinux security advisories (ALSA) and upstream kernel CVEs.
  • Triage CVEs against the published scope: critical, remotely exploitable or actively exploited vulnerabilities.
  • Backport security fixes to the supported AlmaLinux 10 kernels, resolving conflicts across version differences.
  • Generate live patch modules with klp-build. Initially, publish individual CVE patches; evaluate cumulative/atomic replace strategies as the pipeline matures.
  • Identify CVEs that cannot be live patched (e.g. __init changes, data structure layout changes) and record them in the coverage-gap log.

Build and distribution pipeline

  • Reproduce AlmaLinux kernel builds exactly (same source, config and compiler), e.g. in mock with the AlmaLinux buildroot, so modules match the running kernel.
  • Maintain the automated pipeline: CVE scan, build plan, klp-build, RPM packaging, signing, test gate, createrepo_c publication.
  • Plan and execute the move of support to new AlmaLinux kernel releases as they ship.
  • Verify the exact client-side commands work on stock AlmaLinux 10 (dnf kpatch auto plus kpatch-patch-* packages).

Testing and release gate

  • Run the mandatory pre-publication gate on every patch:
  • load and unload on the target kernel in QEMU;
  • run kernel livepatch selftests;
  • for high-severity CVEs: apply and revert under load on a FishOS test cluster with running VMs and Ceph I/O.
  • Document which tests were passed for each release.
  • A patch that passes the gate is cleared for publication. Do not publish a patch that has failed the gate.

Release and withdrawal

  • Publish approved patches with release notes listing the CVEs covered.
  • Publish the source for every released patch alongside its RPM.
  • If a published patch misbehaves: pull it from the repository, publish an advisory, and give users revert instructions.
  • Maintain the runbook and coverage-gap log.

Secure Boot

  • Manage the Sardina module-signing key and RPM signing key.
  • Document and verify mokutil key enrolment for AlmaLinux users.
  • Work with the Ops team to automate key enrolment on FishOS.

Required skills

  • Linux kernel development: strong C, with hands-on experience backporting kernel patches or debugging kernel modules. Screening will test this directly.
  • Live patching: practical experience with klp-build or kpatch, or demonstrated ability to learn them quickly from source.
  • RPM and the RHEL/AlmaLinux ecosystem: kernel packaging (RPM/SRPM), mock, dnf, createrepo_c and repository metadata.
  • Automation: Python or shell; building and maintaining CI-style pipelines.
  • Signing: GPG key management and RPM signing; kernel module signature verification and MOK enrolment.
  • English: working proficiency.

Preferred skills

  • Spoken Russian and/or Mandarin, in addition to English. This is to work closely with Russian- and Mandarin-speaking team members.
  • QEMU-based VM validation or canary testing.
  • Upstream kernel contributions or involvement in the live patching community.
  • Koji or similar build orchestration systems.
  • Ceph I/O testing or storage-heavy workloads.

Key traits

  • Boundary awareness: Willing to say plainly "this CVE needs a reboot" and "this kernel is out of scope" rather than forcing a risky patch
  • Release discipline: Understands that a bad patch is the main reputational risk and will not skip the test gate under pressure
  • Self-directed: Comfortable owning an end-to-end pipeline with minimal supervision
  • Confidentiality: handles embargoed vulnerabilities and unreleased plans with discretion.

Working context

  • Scope: AlmaLinux 10 only, starting with one kernel version.
  • Infrastructure: runs separately from the existing Koji-based OpenStack build infrastructure, but follows the same automation approach.
  • Primary tooling: klp-build, kpatch/kpatch-dnf, mock, createrepo_c, GPG, systemd timers, QEMU and a FishOS test cluster.
  • Release authority: Routine patches are approved by the automated gate; no per-patch sign-off is required. A named release authority reviews weekly summaries, approves exceptions (e.g., a patch that cannot pass the full gate for a documented reason), and handles emergency withdrawals.
  • Backup: one Ops team member is cross-trained on the test gate, publication and withdrawal steps early in the project.

Success Criteria

First milestone

  • klp-build confirmed working on a real Alma 10 kernel
  • End-to-end pipeline producing a signed, tested patch for one CVE class on one kernel version
  • Test gate passed on at least two consecutive builds
  • Coverage-gap log and runbook exist

Second milestone

  • Pipeline operating reliably with the test gate on every build
  • Secure Boot enrollment automated on FishOS
  • Sources and tooling published under the appropriate license
  • Runbook and coverage-gap log maintained and current
Keywords
Linux Kernel DevelopmentC ProgrammingKernel Patch BackportingLinux Live PatchingRPM PackagingAlmaLinuxMockDNFCreaterepo_cPythonShell ScriptingCI Pipeline AutomationGPG Key ManagementRPM SigningQEMU TestingSecure BootLinux KernelAlmaLinux 10OpenStackKubernetes

Interesuje Cię ta oferta?