Healthcare Expert

About the Healthcare Expert Role at Taskify AI

Taskify AI is seeking a highly skilled and experienced Healthcare Expert to join our client, a leading AI research organization. This role offers an hourly contract engagement as an independent contractor, focusing on the evaluation and improvement of advanced conversational AI systems used in healthcare and medical contexts. You will ensure AI responses to clinical and health-related questions are accurate, clear, safe, and adhere to real-world healthcare standards.

Key Responsibilities

• Write and refine prompts that guide AI behavior in healthcare and medical scenarios.
• Evaluate AI-generated responses to healthcare-related queries for accuracy, clarity, reasoning, and completeness.
• Verify all medical claims using trusted public sources and authoritative references.
• Annotate and score model outputs by identifying strengths, weaknesses, and factual or clinical inaccuracies.
• Assess tone, appropriateness, and completeness of responses for real-world healthcare communication.
• Ensure all outputs align with defined system guidelines and conversational standards.
• Apply standardized evaluation frameworks, taxonomies, and benchmarks consistently.

Required Qualifications

• At least 5 years of professional experience in healthcare.
• An advanced healthcare-related degree or credential (e.g., MD, DO, RN, NP, PA, PharmD, MPH, or equivalent).
• Expertise in one or more of the following areas: General clinical care, Specialty medicine or surgery, Diagnostics, imaging, or laboratory medicine, Public health, healthcare systems, or healthcare administration.
• Strong familiarity with large language models (LLMs) and how they are used in practice.
• Excellent written communication skills for complex medical topics.
• Exceptional attention to detail and ability to detect subtle clinical or factual errors.

Preferred Qualifications

• Experience with RLHF, AI evaluation, or data annotation.
• Background in medical writing, editing, or clinical documentation.
• Experience explaining medical information to non-clinical audiences.
• Familiarity with structured evaluation rubrics or quality scoring systems.

What Success Looks Like

In this Healthcare Expert role, success is defined by your consistent ability to identify medical inaccuracies, unsafe reasoning, or unclear explanations in AI outputs. Your valuable feedback will directly contribute to improving the reliability, clarity, and safety of healthcare-related AI systems. You will deliver high-quality, reproducible evaluation data that strengthens overall AI model performance.

Key skills/competency

• Healthcare Expertise
• AI Evaluation
• Medical Accuracy
• Clinical Reasoning
• Prompt Engineering
• Data Annotation
• Large Language Models (LLMs)
• Medical Writing
• Quality Assurance
• Clinical Documentation