AI QA Engineer

About Lenovo

We are Lenovo. We do what we say. We own what we do. We WOW our customers.

Lenovo is a US$69 billion revenue global technology powerhouse, ranked #196 in the Fortune Global 500, and serving millions of customers every day in 180 markets. Focused on a bold vision to deliver Smarter Technology for All, Lenovo has built on its success as the world’s largest PC company with a full-stack portfolio of AI-enabled, AI-ready, and AI-optimized devices (PCs, workstations, smartphones, tablets), infrastructure (server, storage, edge, high performance computing and software defined infrastructure), software, solutions, and services. Lenovo’s continued investment in world-changing innovation is building a more equitable, trustworthy, and smarter future for everyone, everywhere. Lenovo is listed on the Hong Kong stock exchange under Lenovo Group Limited (HKSE: 992) (ADR: LNVGY).

This transformation together with Lenovo’s world-changing innovation is building a more inclusive, trustworthy, and smarter future for everyone, everywhere. To find out more visit www.lenovo.com, and read about the latest news via our StoryHub.

About Our Team

We’re building the next generation of AI-powered experiences, and quality is at the heart of everything we deliver. As an AI QA Engineer, you’ll join a collaborative team of innovators focused on ensuring our AI systems are accurate, safe, and reliable at scale. From large language models to vision systems and agentic workflows, we’re pushing the boundaries of what’s possible—and we need your expertise to make sure every product meets the highest standards.

We recently introduced Qira, our new cross‑device Personal AI that works seamlessly across Lenovo and Motorola products. More information can be found here: https://news.lenovo.com/pressroom/press-releases/lenovo-unveils-lenovo-and-motorola-qira/

Location: Chicago, IL,

Hybrid (3 days on-site, 2 days remote)

What You'll Do

• Design and implement comprehensive test frameworks for AI/ML systems: LLMs, vision models, RAG pipelines, agentic workflows
• Build automated evaluation systems for model outputs—accuracy, relevance, bias detection, hallucination monitoring
• Create quality benchmarks and continuous testing pipelines for generative AI (content quality, safety, brand alignment)
• Develop performance monitoring infrastructure: model drift detection, data quality validation, inference latency tracking
• Establish AI-specific quality gates and work with product/engineering teams in Agile sprints to define acceptance criteria
• Stay ahead of AI testing trends: adversarial testing, red-teaming, AI safety evaluation

Basic Qualifications

• 3+ years in QA/test engineering, with focus on AI/ML systems
• Deep understanding of ML model evaluation: classification metrics, NLP evaluation (BLEU, ROUGE, human eval), vision model testing
• Experience building test automation frameworks (Python, pytest, CI/CD pipelines)

Preferred Qualifications

• Knowledge of AI risks: bias, fairness, safety, hallucinations, adversarial attacks
• Strong collaboration skills—comfortable working with data scientists, ML engineers, and product managers
• Passion for AI quality and keeping current with testing methodologies for generative AI

We are hiring for multiple roles at various levels.

The base salary budgeted range for this position is $110K-$150K. Individuals may also be considered for bonus and/or commission.

Lenovo’s various benefits can be found on www.lenovobenefits.com

We are an Equal Opportunity Employer and do not discriminate against any employee or applicant for employment because of race, color, sex, age, religion, sexual orientation, gender identity, national origin, status as a veteran, and basis of disability or any federal, state, or local protected class.

Key skills/competency

• AI/ML Testing
• Quality Assurance
• LLMs
• Vision Models
• Python
• Pytest
• CI/CD
• Bias Detection
• AI Safety
• Performance Monitoring