English Writing Specialist
Keystone Recruitment
Job Overview
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
Role Overview
One of our clients works with leading AI research teams to improve the quality, reliability, and usefulness of large language models (LLMs). These systems are used across everyday and professional settings, so accurate, clear, and well-reasoned responses are essential.
This English Writing Specialist role focuses on evaluating and improving general conversational AI behavior. You will review model-generated responses, provide expert human feedback, and help ensure AI outputs meet high standards for correctness, tone, and usefulness.
Key Responsibilities
- Evaluate AI-generated responses to user questions
- Perform fact-checking using trusted public sources and tools
- Annotate response strengths, weaknesses, and factual inaccuracies
- Assess reasoning quality, clarity, tone, and completeness
- Ensure outputs follow conversational and safety guidelines
- Apply structured evaluation rubrics and benchmarks
Required Qualifications
- Bachelor’s degree
- Native-level or fluent English
- Experience using large language models (LLMs)
- Excellent written communication and analytical skills
- High attention to detail
- Comfortable working across a wide range of topics
- Background in structured or analytical fields (e.g., research, policy, analytics, linguistics, engineering)
- Strong college-level mathematics skills
Nice to Have
- Experience with RLHF, model evaluation, or data annotation
- Editing, writing, or content review experience
- Ability to compare multiple outputs and make fine-grained quality judgments
- Familiarity with evaluation frameworks and scoring rubrics
What Success Looks Like
- You detect factual mistakes, reasoning gaps, and communication flaws
- Your feedback improves the accuracy and clarity of AI responses
- You produce consistent and reproducible evaluation results
- AI systems become more reliable because of your work
Why Join
One of our clients is building next-generation human-in-the-loop AI systems used globally. This flexible, remote role gives you the opportunity to directly influence how AI communicates with users in English.
Contract & Payment Terms
- Independent contractor
- Fully remote with flexible scheduling
- Weekly payments via Stripe or Wise
- Projects may be extended or concluded based on performance and business needs
- No access to confidential or proprietary employer data
- H1-B and STEM OPT not supported
Key skills/competency
- AI evaluation
- LLM feedback
- Fact-checking
- Content annotation
- Reasoning assessment
- English communication
- Analytical skills
- Attention to detail
- Conversational AI
- Structured evaluation
How to Get Hired at Keystone Recruitment
- Research Keystone Recruitment's clients: Study the type of cutting-edge AI research and language models they support.
- Showcase LLM expertise: Highlight your direct experience using and evaluating large language models in your application.
- Emphasize writing and analytical prowess: Provide concrete examples of clear, concise, and detailed evaluations in your resume.
- Detail fact-checking methodology: Demonstrate your systematic approach to verifying information accuracy using trusted sources.
- Prepare for critical feedback discussion: Be ready to articulate how you provide constructive criticism and improve content quality.
Frequently Asked Questions
Find answers to common questions about this job opportunity
Explore similar opportunities that match your background