Remote Coding Annotator

About TransPerfect and DataForce

DataForce by TransPerfect is a division of the world's largest provider of language and technology solutions, with offices in over 100 cities. They specialize in high-quality data for Human-Machine Interaction, focusing on gathering, enriching, and processing data for Machine Learning across various AI domains.

Role: Remote Coding Annotator

DataForce is actively seeking skilled Software Engineers to join their team as Remote Coding Annotators. This freelance role supports the development and evaluation of advanced AI models. Key responsibilities include creating high-quality coding prompts and answers, benchmarking model performance, and identifying failure cases in both internal and competitor models. Candidates will play a crucial part in building realistic evaluation environments and supporting reinforcement learning workflows.

Key Responsibilities

• Create high-quality coding prompts and reference solutions, adhering to industry benchmarks like SWE-Bench and Terminal-Bench.
• Develop prompts specifically focused on code refactoring, code generation, and complex problem-solving scenarios.
• Evaluate AI model outputs to accurately identify errors, limitations, and recurring failure patterns in reasoning, correctness, and execution.
• Design and maintain robust coding environments essential for effective evaluation and reinforcement learning (RL) pipelines.
• Execute precise coding-specific validation checks using established criteria and specialized tools provided by other annotation teams.
• Document detailed findings, comprehensive evaluation results, and actionable insights to drive model improvement and refine training strategies.
• Perform meticulous code reviews and annotations, ensuring accuracy and full compliance with project standards.
• Work extensively with Python and front-end technologies such as JavaScript and TypeScript, with potential involvement in Java projects.
• Successfully manage and execute repetitive tasks with exceptional precision and maintain consistently high-quality standards.
• Collaborate effectively with cross-functional teams to continuously enhance code quality and documentation practices.

Job Requirements

• Strong proficiency in Python and front-end languages (JavaScript, TypeScript).
• Experience with Java is a significant advantage.
• Demonstrated expertise in code review and quality assurance processes.
• Proven ability to handle repetitive tasks with meticulous attention to detail.
• Minimum of 3 years of professional work experience in a related field.
• Excellent written and verbal communication skills in English.

Education

A Bachelor's degree in Computer Science, Software Engineering, Computer Engineering, or a closely related technical field is required.

Key skills/competency

• Python
• JavaScript
• TypeScript
• Code Annotation
• AI Model Evaluation
• Reinforcement Learning
• Code Refactoring
• Problem Solving
• Quality Assurance
• Technical Analysis
• Documentation