Machine Learning Engineer

About Red Hat

Red Hat is the world's leading provider of enterprise open source software solutions, renowned for its community-powered approach to delivering high-performing Linux, cloud, container, and Kubernetes technologies. Operating in over 40 countries, the company offers flexible work options including in-office, hybrid, and fully remote arrangements.

About the Role

The Machine Learning Engineer will join the AI Inference Engineering team working on distributed vLLM infrastructure within the llm-d project. The role involves developing and optimizing scalable inference systems for large language models (LLMs) using Kubernetes-native deployments and collaborating with an open-source community.

Responsibilities

• Design, develop, and test new features for the AI Inference platform.
• Engage actively with upstream open-source communities.
• Develop distributed inference infrastructure leveraging Kubernetes APIs and operators.
• Implement system components in Go and/or Rust and integrate with vLLM.
• Optimize resource utilization, fault tolerance, and stability of inference systems.
• Participate in technical design discussions and conduct code reviews.

Qualifications

• Proficiency in Python and/or GoLang.
• Experience with cloud-native Kubernetes and service mesh technologies.
• Understanding of networking protocols such as HTTP/2, gRPC, and API gateways.
• Knowledge of LLM serving runtimes including vLLM and TensorRT-LLM.
• Excellent communication skills and ability to work in a dynamic environment.
• Optional: Experience in C, C++, or Rust and familiarity with GPU benchmarking tools.

Benefits

Comprehensive medical, dental, and vision coverage, retirement plans with employer matching, paid time off, parental leave, stock purchase plans, tuition reimbursement, and more.

Key skills/competency

• Python
• GoLang
• Kubernetes
• vLLM
• Cloud-native
• Distributed Systems
• Networking
• Open Source
• Deep Learning
• Fault Tolerance