Evaluation Specialist/Recent Grad

Overview

Responsibilities

• Develop original, challenging question-and-answer pairs that test the limits of advanced AI models across diverse subjects.
• Conduct in-depth research, applying rigorous source triangulation to ensure each answer is accurate, comprehensive, and well-documented.
• Create multi-step questions that require synthesis and analytical reasoning, not just single-source lookups.
• Test your questions against AI models, analyze outcomes, and iterate to increase or adjust difficulty as needed.
• Document your research process, providing clear citations and logical explanations for each answer.
• Refine content based on reviewer input and maintain alignment with project guidelines and quality standards.

Requirements

• Required qualifications: not separately stated in the listing; the qualifications below are labelled preferred.
• English fluency is required, but it does not need to be your first language.

Preferred

• Proven ability to conduct thorough, independent research and critically evaluate information from multiple sources.
• Exceptional attention to detail and written precision in English.
• Talent for crafting nuanced and well-structured questions that probe deep understanding.
• Strong self-direction, reliability, and accountability in remote, independent project work.
• Interest in cutting-edge AI and a passion for testing the boundaries of current model capabilities.
• Prior experience in AI training, evaluation, or content creation is a plus, but not required.
• Open to recent graduates, advanced-degree holders, and individuals from any academic or professional background with demonstrated research and writing expertise.

Engagement and pay

• Role type: contractor, as stated on the source.
• Pay: $20-$60/hour, as listed on the source.
• Compensation is output-based: experts are paid per task that meets the project specifications; time required varies by expert, and minimum weekly submission requirements apply.
• The source states roles are typically filled within 48 hours and selected experts are expected to start their first tasks within 24-48 hours of completing onboarding.

Eligibility

• Country eligibility not stated. Check with the provider before applying.