Software Engineer, AI Research Annotation

About the Software Engineer, AI Research Annotation Role at Keystone Recruitment

Join leading AI labs by applying your software engineering expertise to improve model reasoning. In this role, you will review real-world coding tasks and meticulously document problem-solving processes, directly contributing to the advancement of AI capabilities.

Project Overview

This project supports the development of advanced AI coding capabilities through structured dataset creation. You will work with authentic software engineering scenarios derived from open-source repositories, capturing realistic developer thought processes to inform and refine AI models.

Detailed Responsibilities

• Review AI-generated code trajectories on SWE-bench style tasks, analyzing their efficiency and correctness.
• Reconstruct step-by-step reasoning, thoroughly explaining the solution approaches taken.
• Annotate crucial decision points, effective debugging strategies, and significant technical choices within the code.
• Utilize the full codebase context, including issues, tests, and pull requests, to ensure accurate and comprehensive documentation.
• Ensure all annotations consistently reflect authentic and best-practice software engineering methodologies.

Technical Requirements

• A minimum of 2+ years of hands-on software development experience.
• Strong proficiency in programming languages such as Python, JavaScript/TypeScript, or other similar relevant languages.
• Demonstrated experience in debugging complex codebases and effectively resolving intricate issues.
• Familiarity with standard Git workflows, including pull requests and issue tracking systems.
• Exceptional ability to clearly and concisely articulate technical reasoning in written form.

Project Structure & Logistics

• Duration: Initially 1-2 months, with strong potential for extension based on performance and project needs.
• Start: Immediate onboarding available for qualified candidates.
• Onboarding Process: Includes a resume review followed by a brief 15-minute AI technical interview.
• Pace: Self-scheduled work within a committed weekly minimum of 15-25 hours, with flexibility to scale up to 40 hours.
• Payment: Weekly payments processed conveniently via Stripe/Wise.

Ideal Background

• A Bachelor's degree or higher in Computer Science, Software Engineering, or a closely related field.
• Prior experience with open-source contributions or in-depth repository analysis is highly valued.
• A solid understanding of software development lifecycles (SDLC) and various testing methodologies.
• Comfort and proficiency in working with technical documentation and detailed specifications.

Success Profile

Top performers in this role are recognized with rate increases and receive priority consideration for future AI research projects. This is an opportunity to join a cohort of experienced engineers dedicated to shaping the next-generation of coding AI.

Application Process

To apply, upload your resume and complete a brief technical screening. Keystone Recruitment welcomes all qualified software engineers, regardless of their background, to contribute to this exciting project.

Key skills/competency

• Software Engineering
• AI Research
• Code Annotation
• Python
• JavaScript/TypeScript
• Debugging
• Git Workflows
• Problem Solving
• Technical Documentation
• Open Source