Staff ML Performance Engineer (Training Efficiency)
Wayve · Sunnyvale, CA
- On site
- Full-time
- $359,000 / year
- Sunnyvale, CA
Email the hiring manager to get a response.
Get their verified email + an intro that's ready to send.
Subject: Interested in the Staff ML Performance Engineer (Training Efficiency) role at Wayve
Hi Avery — I came across the Staff ML Performance Engineer (Training Efficiency) opening and wanted to reach out directly. I've spent the last few years doing exactly this kind of work, and Wayve stood out because…
✎ Personalized to your résumé after sign-up.
- ✓ Verified email of the hiring manager
- ✓ Intro email personalized to your résumé
- ✓ $9/mo = unlimited — any job link
Secure checkout · cancel anytime
Job highlights
- Optimize large-scale ML training jobs for faster model scaling.
- Increase efficiency of training and inference workloads.
- Profile ML workloads to identify performance bottlenecks.
- Design and implement efficiency improvements and observability tools.
- Collaborate with research teams on performance optimization.
About the role
About Us
Founded in 2017, Wayve is the leading developer of Embodied AI technology. Our advanced AI software and foundation models enable vehicles to perceive, understand, and navigate any complex environment, enhancing the usability and safety of automated driving systems.
Our vision is to create autonomy that propels the world forward. Our intelligent, mapless, and hardware-agnostic AI products are designed for automakers, accelerating the transition from assisted to automated driving.
In our fast-paced environment big problems ignite us—we embrace uncertainty, leaning into complex challenges to unlock groundbreaking solutions. We aim high and stay humble in our pursuit of excellence, constantly learning and evolving as we pave the way for a smarter, safer future.
At Wayve, your contributions matter. We value diversity, embrace new perspectives, and foster an inclusive work environment; we back each other to deliver impact. Make Wayve the experience that defines your career!
The role
We are looking for a Staff ML Performance Engineer to join our Training Tech team working on optimizing large scale ML jobs to enable scaling our models to the next order of magnitude. A successful candidate will increase efficiency of training and inference workloads in order to allow Wayve to train larger models faster.
Key Responsibilities
- Profile ML workloads to identify their bottlenecks, e.g. using NVIDIA Nsight Systems
- Design and implement efficiency improvements to maximize MFU and throughput, e.g. parallelism, model compilation, mixed precision
- Design and implement observability tools to identify bottlenecks and drive performance improvements, e.g. to track MFU, throughput, latency, etc
- Design and implement benchmarking tools, e.g. to track efficiency gains or regressions
- Collaborate closely with Research teams to integrate training efficiency improvements and create a culture of performance optimization
About you
In order to set you up for success in this role, we’re looking for the following skills and experience.
Essential
- 10+ years of industry experience driving performance engineering across ML systems, GPU compute infrastructure, distributed platforms or similar field.
- Experience optimizing large scale jobs on GPU compute clusters.
- Experience in working in platform teams and working with research teams.
- Experience in writing, reporting, and tracking performance benchmarks in an open and accessible way.
- Ability to write high quality, well-structured and tested Python code
- BS or MS in Machine Learning, Computer Science, Engineering, or a related technical discipline or equivalent experience
Desirable
- Experience working with concurrent, parallel and distributed computing.
- Experience using NVIDIA NSight Systems or other system profilers.
- Experience implementing GPU kernels (CUDA, Triton, etc).
- Knowledge of computing fundamentals - what makes code fast, secure and reliable.
Compensation & Benefits
This role is a full-time role based in Sunnyvale, CA (hybrid) and the reasonably estimated salary for this role ranges from $336,400 to $359,000, plus a competitive equity package. Actual compensation is based on the candidate's skills, qualifications, and experience.
Wayve's Commitment to Inclusion
Wayve is committed to creating an inclusive interview experience. If you require any accommodations or adjustments to participate fully in our interview process, please let us know.
We understand that everyone has a unique set of skills and experiences and that not everyone will meet all of the requirements listed above. If you’re passionate about self-driving cars and think you have what it takes to make a positive impact on the world, we encourage you to apply.
At Wayve we're committed to creating a diverse, fair and respectful culture that is inclusive of everyone based on their unique skills and perspectives, and regardless of sex, race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership status, sexual orientation, gender identity, veteran status, pregnancy or related condition (including breastfeeding) or any other basis as protected by applicable law.
For more information visit Careers at Wayve. To learn more about what drives us, visit Values at Wayve
For US candidates only, please visit E-Verify Notice and Participation and Right to Work
Disclaimer
We will not ask about marriage or pregnancy, care responsibilities or disabilities in any of our job adverts or interviews. However, we do look to capture information about care responsibilities, and disabilities among other diversity information as part of an optional DEI Monitoring form to help us identify areas of improvement in our hiring process and ensure that the process is inclusive and non-discriminatory.
Key skills/competency
- Machine Learning Performance
- ML Systems Optimization
- GPU Compute Infrastructure
- Distributed Systems
- Python
- Performance Benchmarking
- NVIDIA Nsight Systems
- CUDA
- Triton
- Research Collaboration
Skills & topics
- Staff ML Performance Engineer
- Machine Learning
- Performance Engineering
- ML Optimization
- GPU
- Distributed Systems
- Python
- Training Efficiency
- Automated Driving
- AI
How to get hired
- Tailor your resume: Highlight your 10+ years of experience in ML performance, GPU clusters, and distributed systems, showcasing Python proficiency and benchmark reporting.
- Showcase ML optimization skills: Detail your experience with NVIDIA Nsight Systems, CUDA, or Triton, and any contributions to platform or research teams.
- Quantify your impact: Provide examples of how you've increased MFU, throughput, or reduced latency in previous roles.
- Prepare for technical questions: Be ready to discuss ML workload profiling, bottleneck identification, and performance benchmarking strategies.
- Understand Wayve's mission: Align your application with Wayve's vision for Embodied AI and automated driving.
Technical preparation
Behavioral questions
Frequently asked questions
- What is the salary range for the Staff ML Performance Engineer role at Wayve?
- The estimated salary range for the Staff ML Performance Engineer position at Wayve is $336,400 to $359,000 annually, plus a competitive equity package. The final compensation will be determined by your skills, qualifications, and experience.
- Does Wayve offer remote work options for this Staff ML Performance Engineer position?
- This Staff ML Performance Engineer role is based in Sunnyvale, CA and is a hybrid position. While not fully remote, it offers a blend of in-office and remote work.
- What are the essential qualifications for the Staff ML Performance Engineer job at Wayve?
- Essential qualifications include over 10 years of industry experience in ML performance engineering, optimizing large-scale jobs on GPU clusters, experience with platform and research teams, strong Python coding skills, and a BS or MS in a related technical field or equivalent experience.
- How can I best highlight my qualifications for the Staff ML Performance Engineer role?
- To best highlight your qualifications, emphasize your experience in profiling ML workloads, designing efficiency improvements, implementing observability and benchmarking tools, and collaborating with research teams. Quantify your achievements in increasing MFU, throughput, or reducing latency.
- What kind of ML optimization experience is desirable for this role at Wayve?
- Desirable experience includes working with concurrent, parallel, and distributed computing, using system profilers like NVIDIA NSight Systems, implementing GPU kernels (CUDA, Triton), and a strong understanding of computing fundamentals for performance, security, and reliability.
- What is Wayve's approach to diversity and inclusion in the hiring process?
- Wayve is committed to creating a diverse, fair, and respectful culture and an inclusive interview experience. They encourage applications from individuals with diverse skills and perspectives and offer accommodations if needed.
Similar roles
Open positions we recommend based on this role.