C++/Python Developer

Job Overview

We are looking for skilled C++/Python Developers at Junior, Mid-Level, and Senior levels to contribute to the development, testing, optimization, and performance engineering of deep learning frameworks and AI accelerator software.

The role focuses on PyTorch framework integration, accelerator backend development, automated testing, debugging, and performance optimization. Depending on experience and team allocation, selected candidates may also work on large language model (LLM) inference engines, serving optimization, and benchmarking.

Important: Hands-on development experience with the Gaudi software stack or the Gaudi–PyTorch bridge is mandatory. Candidates without practical development experience in this area will not be considered.

Key Responsibilities

• Develop, debug, test, and optimize C++ and Python components within deep learning frameworks and AI accelerator software stacks.
• Work on PyTorch integration, backend compatibility, operator implementation, and framework-level testing.
• Develop and maintain automated tests using Python, pytest, and PyTorch testing utilities.
• Investigate software defects, reproduce issues, perform root-cause analysis, and implement reliable fixes.
• Contribute to continuous integration and delivery pipelines using GitHub Actions and Jenkins.
• Work with Linux, Docker, Git, and related development and debugging tools.
• Analyze performance and correctness issues across supported hardware and software configurations.
• Contribute code changes, upstream pull requests, technical documentation, and design proposals as appropriate to experience level.
• Collaborate with framework, infrastructure, and performance engineering teams to improve software quality, reliability, and execution efficiency.

Mandatory Technical Skills

• Hands-on development experience with the Gaudi software stack and/or the Gaudi– Pytorch bridge.
• Strong programming skills in C++ and/or Python, depending on the role's seniority.
• Experience with Python testing frameworks, particularly pytest.
• Familiarity with PyTorch testing utilities, including common device type, instantiate device type tests, and common utils.
• Experience working in Linux environments and using Git for version control.
• Experience with CI pipelines using GitHub Actions and/or Jenkins.
• Practical debugging skills, including the use of git bisect to identify regressions.
• Understanding of deep learning concepts and PyTorch framework fundamentals.

Additional Requirements for LLM Inference / vLLM Positions

These requirements apply to positions allocated to the LLM inference and serving workstream, which represents approximately 50% of the team.

• Hands-on development experience with vLLM internals, including its engine, scheduler, KV cache, Paged Attention, attention backends, model runner, or hardware platform layer.
• Experience optimizing LLM inference performance, including quantization techniques such as FP8 and INT4.
• Understanding of tensor parallelism, pipeline parallelism, and continuous batching.
• Experience benchmarking LLM serving performance, throughput, latency, and output accuracy.
• Familiarity with Dynamo benchmark suites and model evaluation using Hugging Face, TIMM, or Torch Bench is an advantage.
• Contributions to the vLLM open-source project are highly desirable.

Experience Levels and Expectations

Junior Developer

• Good programming fundamentals in C++ and/or Python.
• Basic understanding of Linux, Git, deep learning, and PyTorch.
• Ability to reproduce bugs, fix tests, and implement simple operators or code changes.
• Willingness to learn framework internals and work under the guidance of senior engineers.

Mid-Level Developer

• Strong C++ and Python programming skills.
• Experience with PyTorch internals or another deep learning framework.
• Foundational knowledge of GPU or accelerator programming, such as CUDA, SYCL, or OpenCL.
• Ability to debug, profile, and optimize framework components.
• Capability to own medium-complexity tasks independently, from investigation through implementation, testing, and upstream pull requests.

Senior Developer

• 6+ years of relevant software development experience.
• Deep understanding of Py Torch internals, including ATen, dispatcher, autograd, torch.compile, Inductor, and custom or out-of-tree backends such as PrivateUse1.
• Strong experience in GPU/accelerator kernel development, profiling, and performance optimization.
• Proven ability to independently deliver complex framework-level changes and large-scale technical tasks.
• Experience contributing to open-source projects, including upstream pull requests.
• Ability to lead technical discussions, prepare design proposals, review code, mentor engineers, and coordinate technical delivery across the team.

What Makes This Opportunity Interesting?

• Work on deep learning framework development and AI accelerator software.
• Solve challenging problems involving framework integration, software correctness, debugging, and performance optimization.
• Contribute to modern AI infrastructure and, where applicable, LLM inference and serving technologies.
• Collaborate with engineers across framework development, testing, and performance engineering.
• Make meaningful contributions to software quality, scalability, and open-source development.