Math Expert

Role: Math Expert

Keystone Recruitment is seeking a remote Math Expert for one of their clients, a leading AI research organization. This pivotal role focuses on enhancing the quality, usefulness, and reliability of general-purpose conversational AI systems, particularly in mathematics-related applications. The goal is to ensure these AI systems demonstrate rigorous formal reasoning, conceptual clarity, and mathematical correctness across foundational and advanced mathematical domains.

What You’ll Do

• Write and refine prompts to guide AI model behavior in mathematical contexts.
• Evaluate large language model (LLM) responses to mathematics-related queries for correctness, rigor, and logical coherence.
• Verify mathematical claims, derivations, and proofs using deep domain expertise.
• Conduct fact-checking using authoritative public sources and mathematical knowledge.
• Annotate model responses by identifying strengths, weaknesses, and factual or conceptual errors.
• Assess clarity, structure, and suitability of explanations for different audience levels.
• Ensure outputs align with expected conversational standards and evaluation guidelines.
• Apply consistent evaluation frameworks, benchmarks, and taxonomies across tasks.

Who You Are

• PhD in Mathematics or a closely related field.
• Demonstrated expertise in Probability & Statistics, with experience in one or more of the following areas: Algebra & Number Theory, Calculus & Analysis, Geometry & Topology, Discrete Mathematics, Logic & Computation.
• Comfortable working with large language models and understanding real-world usage patterns.
• Excellent written communication skills with the ability to explain complex concepts clearly.
• Strong attention to detail and ability to identify subtle logical or conceptual issues.
• Experience reviewing, editing, or evaluating technical or academic writing.

Nice-to-Have Specialties

• Prior experience with RLHF, model evaluation, or data annotation.
• Teaching, mentoring, or explaining mathematics to non-expert audiences.
• Familiarity with structured evaluation rubrics, benchmarks, or review frameworks.

What Success Looks Like

• You consistently identify inaccuracies or weak reasoning in mathematical AI outputs.
• Your feedback improves rigor, correctness, and clarity of model responses.
• You deliver reliable, reproducible evaluation artifacts that strengthen AI performance.
• The client’s AI systems are trusted in mathematical contexts due to your rigorous evaluations.

Why Join This Project

This opportunity allows mathematicians to apply deep theoretical expertise to the evaluation and improvement of cutting-edge AI systems. The role is fully remote and flexible, enabling you to directly influence how mathematical reasoning is represented and communicated at scale.

Contract and Payment Terms

• Engagement as an independent contractor.
• Fully remote with flexible scheduling.
• Projects may be extended, shortened, or concluded based on performance and project needs.
• Work will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are issued weekly via Stripe or Wise based on services rendered.
• H1-B and STEM OPT candidates cannot be supported at this time.

Key skills/competency

• Advanced Mathematics
• AI Model Evaluation
• Large Language Models (LLM)
• Formal Reasoning
• Probability & Statistics
• Algebra & Number Theory
• Calculus & Analysis
• Data Annotation
• Technical Writing
• Logical Coherence