English Writing Specialist
Keystone Recruitment
Job Overview
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
English Writing Specialist at Keystone Recruitment
One of our clients works with leading AI research teams to improve the quality, reliability, and usefulness of large language models (LLMs). These systems are used across everyday and professional settings, so accurate, clear, and well-reasoned responses are essential. This role focuses on evaluating and improving general conversational AI behavior. You will review model-generated responses, provide expert human feedback, and help ensure AI outputs meet high standards for correctness, tone, and usefulness.
Key Responsibilities
- Evaluate AI-generated responses to user questions
- Perform fact-checking using trusted public sources and tools
- Annotate response strengths, weaknesses, and factual inaccuracies
- Assess reasoning quality, clarity, tone, and completeness
- Ensure outputs follow conversational and safety guidelines
- Apply structured evaluation rubrics and benchmarks
Required Qualifications
- Bachelor’s degree
- Native-level or fluent English
- Experience using large language models (LLMs)
- Excellent written communication and analytical skills
- High attention to detail
- Comfortable working across a wide range of topics
- Background in structured or analytical fields (e.g., research, policy, analytics, linguistics, engineering)
- Strong college-level mathematics skills
Nice to Have
- Experience with RLHF, model evaluation, or data annotation
- Editing, writing, or content review experience
- Ability to compare multiple outputs and make fine-grained quality judgments
- Familiarity with evaluation frameworks and scoring rubrics
What Success Looks Like
- You detect factual mistakes, reasoning gaps, and communication flaws
- Your feedback improves the accuracy and clarity of AI responses
- You produce consistent and reproducible evaluation results
- AI systems become more reliable because of your work
Why Join
One of our clients is building next-generation human-in-the-loop AI systems used globally. This flexible, remote role gives you the opportunity to directly influence how AI communicates with users in English.
Contract & Payment Terms
- Independent contractor
- Fully remote with flexible scheduling
- Weekly payments via Stripe or Wise
- Projects may be extended or concluded based on performance and business needs
- No access to confidential or proprietary employer data
- H1-B and STEM OPT not supported
Key skills/competency
- AI Evaluation
- Large Language Models (LLMs)
- Written Communication
- Analytical Skills
- Fact-checking
- Content Review
- Linguistics
- Data Annotation
- Reasoning Assessment
- Conversational AI
How to Get Hired at Keystone Recruitment
- Research Keystone Recruitment's culture: Study their mission, values, recent news, and employee testimonials on LinkedIn and Glassdoor.
- Tailor your resume: Highlight native English, LLM experience, and strong analytical/writing skills for the English Writing Specialist role.
- Showcase your expertise: Provide examples of fact-checking, content review, or data annotation in your application.
- Prepare for skill assessments: Expect tasks evaluating your writing, critical thinking, and LLM interaction abilities.
- Demonstrate remote work readiness: Emphasize your ability to work independently and manage flexible schedules effectively.
Frequently Asked Questions
Find answers to common questions about this job opportunity
Explore similar opportunities that match your background