Healthcare Expert

Overview

The Healthcare Expert role at Keystone Recruitment is a remote, hourly contractor engagement. This position focuses on evaluating and improving advanced conversational AI systems used in healthcare, ensuring that responses to clinical and health-related questions are accurate, clear, safe, and aligned with real-world healthcare standards.

Key Responsibilities

• Write and refine prompts to guide AI behavior in healthcare scenarios.
• Evaluate AI-generated responses for accuracy, clarity, reasoning, and completeness.
• Verify medical claims using trusted public sources and authoritative references.
• Annotate and score model outputs, identifying strengths and weaknesses.
• Assess tone, appropriateness, and adherence to guidelines.
• Apply standardized evaluation frameworks, taxonomies, and benchmarks.

Required Qualifications

• Minimum 5 years professional experience in healthcare.
• Advanced healthcare-related degree or credential (e.g., MD, DO, RN, NP, PA, PharmD, MPH).
• Expertise in clinical care, specialty medicine, diagnostics, public health, or healthcare administration.
• Familiarity with large language models (LLMs) in practice.
• Excellent written communication on complex medical topics.
• Exceptional attention to detail in detecting clinical or factual errors.

Preferred Qualifications

• Experience with RLHF, AI evaluation, or data annotation.
• Background in medical writing, editing, or clinical documentation.
• Experience explaining medical information to non-clinical audiences.
• Familiarity with structured evaluation rubrics or quality scoring systems.

What Success Looks Like

• Identifying medical inaccuracies, unsafe reasoning, or unclear explanations consistently.
• Providing feedback that enhances reliability, clarity, and safety of AI outputs.
• Delivering reproducible evaluation data that strengthens AI model performance.

Contract & Payment Terms

• Independent contractor engagement with remote work and flexible scheduling.
• Weekly payments via Stripe or Wise.
• Project scope and duration vary based on performance and client needs.
• No access to confidential or proprietary employer data required.
• H1-B and STEM OPT sponsorship are not available.

Application Process

Submit your resume for review. Selected candidates will complete a short healthcare-focused evaluation or screening form.

Key skills/competency

• Healthcare
• AI
• Clinical
• Evaluation
• Annotation
• Medical Writing
• Data Analysis
• Prompt Engineering
• Attention to Detail
• Independent Contractor