Python Expert - AI Evaluator

@ Braintrust

Project Description

This project involves reviewing AI-generated responses, evaluating technical accuracy and best practices, and providing expert feedback to train AI systems to write better code. You will work with various programming languages and coding challenges ensuring that the AI outputs are functional, efficient, and adhere to industry standards.

Compensation

Compensation is competitive and varies based on education level, institution, and prior experience.

Hiring Profile

• BS, MS, or PhD with a significant focus on Computer Science (Required).
• Expert in Python with strong grasp of algorithms, data structures, and debugging practices.
• Expert-level English with excellent writing and fact-checking skills.
• Experience using large language models (LLMs) and, preferably, RLHF annotation.
• Prior Software Engineering experience is strongly preferred.
• Familiarity with additional languages such as C#, Java, SQL, C++, TypeScript, PHP, C, Go, Bash, PowerShell, Rust, or R is a plus.

Application & Onboarding Process

Apply via the partner site. Braintrust will send an email with detailed instructions. A qualification exam (1–2 hours) and a project exam (1–2 hours) will assess your role-specific expertise and adherence to guidelines, with compensation provided upon passing each.

Commitment

Minimum commitment is 17–20 hours per week, with a recommended commitment of 8 hours per day based on project scope and timelines.

Key skills/competency

• Python
• Algorithms
• Data Structures
• Debugging
• LLM
• RLHF
• Software Engineering
• Technical Evaluation
• Feedback
• Multi-language