Performance Lead AI Agent

@ Google

Overview

The Performance Lead AI Agent at Google drives data-driven decision-making to enhance the creator experience on YouTube. You will translate complex conversational and performance data into actionable insights, developing new measurement frameworks for AI operations and creating KPIs for generative AI products.

Minimum Qualifications

• Bachelor’s degree or equivalent practical experience.
• 5 years of experience in an analytical role in customer operations or for an AI/ML product.
• 5 years of experience manipulating datasets and conducting analyses using SQL and Python.
• Experience creating executive communications and presenting to non-technical audiences.

Preferred Qualifications

• Master’s degree in a relevant field such as Business, Statistics, or Engineering.
• Experience in data analysis and modeling for AI/ML products.
• Expertise in developing KPIs and measurement frameworks for LLMs or generative AI.
• Familiarity with experimental design (A/B testing) in conversational AI or user-facing ML products.
• Strong project management and communication skills.

Responsibilities

• Establish and manage core KPIs for the AI Agent including LLM-based metrics.
• Analyze performance and conversational data to identify optimization opportunities.
• Design and interpret A/B tests on prompts and conversational flows.
• Build forecasting models to evaluate new AI capabilities.
• Translate analytical findings into actionable insights for cross-functional teams.

Key skills/competency

• Data Analysis
• SQL
• Python
• Forecasting
• A/B Testing
• KPIs
• Executive Communication
• Predictive Modeling
• AI/ML
• Project Management