Healthcare Expert

About the Role: Healthcare Expert at Keystone Recruitment

Keystone Recruitment is seeking a highly skilled Healthcare Expert to join an hourly contract engagement. This remote role involves collaborating with a leading AI research organization to enhance advanced conversational AI systems used in critical healthcare and medical contexts. You will play a pivotal role in ensuring AI responses are accurate, clear, safe, and adhere to real-world healthcare standards.

Key Responsibilities

• Write and refine prompts that effectively guide AI behavior in complex healthcare and medical scenarios.
• Rigorously evaluate AI-generated responses to health-related queries for accuracy, clarity, logical reasoning, and comprehensive completeness.
• Verify all medical claims through diligent research using trusted public sources and authoritative references.
• Annotate and score model outputs, meticulously identifying strengths, weaknesses, and any factual or clinical inaccuracies.
• Assess the tone, appropriateness, and completeness of AI responses to ensure effective real-world healthcare communication.
• Ensure all AI outputs strictly align with defined system guidelines and established conversational standards.
• Apply standardized evaluation frameworks, taxonomies, and benchmarks consistently to maintain data quality.

Required Qualifications

• At least 5 years of professional experience in the healthcare sector.
• Possession of an advanced healthcare-related degree or credential (e.g., MD, DO, RN, NP, PA, PharmD, MPH, or equivalent).
• Demonstrated expertise in one or more of the following critical areas: General clinical care, Specialty medicine or surgery, Diagnostics, imaging, or laboratory medicine, Public health, healthcare systems, or healthcare administration.
• Strong familiarity with large language models (LLMs) and their practical application.
• Exceptional written communication skills, particularly for conveying complex medical topics.
• Outstanding attention to detail and a proven ability to detect subtle clinical or factual errors.

Preferred Qualifications

• Experience with Reinforcement Learning from Human Feedback (RLHF), AI evaluation methodologies, or data annotation.
• A background in medical writing, editing, or clinical documentation.
• Proven experience in explaining complex medical information to non-clinical audiences effectively.
• Familiarity with structured evaluation rubrics or quality scoring systems.

What Success Looks Looks Like

• Consistently identifying medical inaccuracies, unsafe reasoning, or unclear explanations within AI outputs.
• Your constructive feedback demonstrably improves the reliability, clarity, and safety of healthcare-related AI outputs.
• You deliver high-quality, reproducible evaluation data that significantly strengthens AI model performance.

Contract & Payment Terms

• Independent contractor engagement.
• Fully remote position with flexible scheduling options.
• Weekly payments processed via Stripe or Wise.
• Project scope and duration are adaptable based on performance and client needs.
• No access to confidential or proprietary employer data is required.
• H1-B and STEM OPT sponsorship is not available for this role.

Key skills/competency

• Healthcare Expertise
• AI Evaluation
• Clinical Reasoning
• Medical Accuracy
• LLM Familiarity
• Prompt Engineering
• Data Annotation
• Regulatory Compliance
• Written Communication
• Attention to Detail