ML Engineer, Foundation Model Evaluation

About Waymo

Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The World's Most Experienced Driver™—to improve access to mobility while saving thousands of lives now lost to traffic crashes. The Waymo Driver powers Waymo’s fully autonomous ride-hail service and can also be applied to a range of vehicle platforms and product use cases. The Waymo Driver has provided over ten million rider-only trips, enabled by its experience autonomously driving over 100 million miles on public roads and tens of billions in simulation across 15+ U.S. states.

About the AI Foundations Team

The mission of the Waymo AI Foundations team is to develop machine learning solutions addressing open problems in autonomous driving, towards the goal of safely operating Waymo vehicles in dozens of cities and under all driving conditions. As part of our work, we also initiate and foster collaborations with other research teams in Alphabet. AI Foundations areas that we are currently focusing on include reinforcement learning, learning from demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust evaluation.

The Role: ML Engineer, Foundation Model Evaluation

This role follows a hybrid work schedule and you will report to a Senior Research Scientist.

Responsibilities

• Develop and extend cutting-edge research in robotics and machine learning to advance state-of-the-art methodologies for evaluating the quality, safety, and realism of embodied AI agents.
• Partner within and across organizations to land disruptive and innovative tech in production.
• Work with a variety of state-of-the-art Foundation Models.
• Drive model development through defining evaluation and benchmarks.
• Implement and extend large-scale data and evaluation pipelines.

Minimum Qualifications

• Masters degree in Computer Science, Machine Learning, Robotics, similar technical field of study, or equivalent practical experience.
• Proficiency in Python.
• Familiarity with one of the modern deep learning frameworks (e.g. Pytorch, JAX, Tensorflow).
• Prior work in an industrial or research setting developing methodologies for the evaluation of ML models.

Preferred Qualifications

• Strong hands-on SWE skills, able to design, implement, and extend large distributed pipelines.
• Track record of publications in top-tier conferences or leading open source projects in the related fields.
• Proficiency in C++.
• Experience in AV planning and related research.
• Experience in labeling and curating data for ML eval and training.

Key skills/competency

• Machine Learning
• Model Evaluation
• Robotics
• Python
• Deep Learning Frameworks
• Autonomous Driving
• Data Pipelines
• Foundation Models
• AI Agents
• Research & Development