Principal Software Engineer Llama Stack and Testing

Principal Software Engineer - Llama Stack and Testing

At Red Hat, our commitment to open source innovation extends beyond our products; it’s embedded in how we work and grow. Red Hatters embrace change and have a strong growth mindset. We encourage our teams to proactively, thoughtfully, and ethically use AI to simplify workflows, cut complexity, and boost efficiency, empowering associates to focus on higher-impact work and create innovative solutions.

Red Hat’s Global Engineering team is looking for a Principal Software Engineer to join the Agentic and AI Engineering Tools team. You will develop new APIs, tooling, providers, and improve overall code quality in the upstream Llama Stack project. This role directly contributes to Red Hat’s AI/ML products, focusing on the design, development, and refinement of software features that enable Llama Stack for Enterprise readiness.

The ideal candidate has a proven background in developing robust and scalable code. Responsibilities include adhering to coding best practices, writing well-documented and efficient code, building and implementing upstream unit and E2E automated tests, maintaining documentation and comments, following security best practices, participating in code reviews, and staying current with software engineering technologies.

What you will do

• Collaborate with cross-functional teams (Staff Engineers, Engineering, Product Management, User Experience) to define customer needs and ensure product and testing meet requirements.
• Develop and implement comprehensive unit, integration, and E2E tests with Quality Engineers to ensure reliability and performance in the upstream project, maintaining CI/CD workflows in GitHub and ensuring downstream quality.
• Participate in AI-assisted code reviews to identify bugs, security vulnerabilities, and ensure adherence to coding standards, enhancing the review process.
• Leverage AI-driven testing frameworks to automate test case generation, optimize execution, identify edge cases, and improve test coverage and reliability.
• Create and maintain clear, concise upstream technical documentation, including API references and user guides, collaborating with internal tech writers for downstream documentation.

What you will bring

• 10 years of advanced Python and 5+ years Bash/Shell development experience.
• Mandatory experience in testing, automation, and quality engineering.
• Advanced knowledge in designing robust and scalable software for highly scaled and performant Distributed Systems.
• Experience with building agentic/RAG applications, MCP integrations, or developing with LLMs.
• Advanced knowledge in creating automation for GitHub using GitHub Actions or related continuous integration tools.
• Advanced knowledge in developing unit, functional, and end-to-end (E2E) test cases and automation, including frameworks like PyTest.
• Knowledge of Kubernetes/OpenShift and operational knowledge of building/deploying containers.
• Experience developing, deploying, or maintaining On-prem or Cloud Infrastructure.
• Ability to quickly learn and use new tools and technologies.

Nice to Haves

• Experience with Security, Observability, Performance, or Scale.
• Understanding of DevOps methodology, scrum, and/or Jira.
• Experience with AI and Machine Learning platforms, tools, and frameworks such as LlamaStack, LangChain, PyTorch, LLaMA.cpp, vLLM, LangGraph, and Kubeflow.

About Red Hat

Red Hat is the world’s leading provider of enterprise open source software solutions. Our associates work flexibly across various environments, from in-office to fully remote, depending on role requirements. We encourage bringing your best ideas, regardless of title or tenure, fostering an open and inclusive environment where innovation thrives.

Key skills/competency

• Principal Software Engineer
• Llama Stack
• AI/ML
• Python
• Bash/Shell
• Testing Automation
• Distributed Systems
• Kubernetes/OpenShift
• CI/CD
• API Development