Healthcare Expert

Role Overview: Healthcare Expert

Keystone Recruitment's client, a leading AI research organization, is seeking a Healthcare Expert for an hourly contract. This fully remote role involves evaluating and enhancing advanced conversational AI systems used in healthcare and medical contexts. You will be instrumental in ensuring AI outputs are accurate, clear, safe, and adhere to real-world healthcare standards, focusing on responses to clinical and health-related questions.

Key Responsibilities

• Write and refine prompts to guide AI behavior in healthcare and medical scenarios.
• Evaluate AI-generated responses for accuracy, clarity, reasoning, and completeness.
• Verify all medical claims using trusted public sources and authoritative references.
• Annotate and score model outputs, identifying strengths, weaknesses, and clinical inaccuracies.
• Assess tone, appropriateness, and completeness of responses for real-world healthcare communication.
• Ensure all outputs align with defined system guidelines and conversational standards.
• Apply standardized evaluation frameworks, taxonomies, and benchmarks consistently.

Required Qualifications

• At least 5 years of professional experience in healthcare.
• An advanced healthcare-related degree or credential (e.g., MD, DO, RN, NP, PA, PharmD, MPH, or equivalent).
• Expertise in one or more of the following areas: General clinical care, Specialty medicine or surgery, Diagnostics, imaging, or laboratory medicine, Public health, healthcare systems, or healthcare administration.
• Strong familiarity with large language models (LLMs) and their practical applications.
• Excellent written communication skills for complex medical topics.
• Exceptional attention to detail and ability to detect subtle clinical or factual errors.

Preferred Qualifications

• Experience with RLHF, AI evaluation, or data annotation.
• Background in medical writing, editing, or clinical documentation.
• Experience explaining medical information to non-clinical audiences.
• Familiarity with structured evaluation rubrics or quality scoring systems.

What Success Looks Like

• You consistently identify medical inaccuracies, unsafe reasoning, or unclear explanations.
• Your feedback improves the reliability, clarity, and safety of healthcare-related AI outputs.
• You deliver high-quality, reproducible evaluation data that strengthens AI model performance.

Contract & Payment Terms

• Independent contractor engagement.
• Fully remote with flexible scheduling.
• Weekly payments via Stripe or Wise.
• Project scope and duration may vary based on performance and client needs.
• No access to confidential or proprietary employer data is required.
• H1-B and STEM OPT sponsorship is not available.

Application Process

• Submit your resume for review.
• Selected candidates will complete a short healthcare-focused evaluation or screening form.

Key skills/competency

• Healthcare Standards
• Medical Claim Verification
• AI Evaluation
• Large Language Models (LLMs)
• Clinical Accuracy
• Medical Writing
• Data Annotation
• Prompt Engineering
• Public Health Expertise
• Attention to Detail