This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an AI Interaction Evaluator based in the United States.

This is a senior-level contract opportunity focused on evaluating how modern AI coding agents interact with experienced software engineers.
Rather than developing production software, you’ll use your engineering expertise to assess the quality, usefulness, and judgment demonstrated by AI-generated responses.
You’ll evaluate tools such as Codex, Claude Code, and Cursor across realistic software-development scenarios.
The role goes beyond checking whether code is technically correct, focusing on reasoning, explanations, developer guidance, and overall interaction quality.
You’ll apply rigorous engineering judgment to distinguish genuinely strong AI assistance from responses that are merely plausible or syntactically correct.
Your feedback will help establish clearer standards for what excellent AI-assisted development should look like.
The engagement is fully remote, flexible, and designed for experienced engineers who want to contribute directly to the evolution of AI coding tools.

AI Interaction Evaluator (Codex / Claude Code, up to $200/hr)

What the job really is