JobFetch

AI Quality Analyst - English

Pay
Pay not listed
Where
Remote
Type
Contract
Eligibility
Country eligibility not stated. Check with the provider before applying.
Listing
Last verified Oct 6, 2026

Apply opens the provider’s application page. You apply there yourself: we never submit applications, and opening the page doesn’t mean you’ve applied. Nobody can promise you’ll be hired.

Some Apply links are referral links. If you apply through them, the provider may pay us a commission. Referral disclosure

Explore weekly job alerts for your skills

Overview

Evaluate a new personalisation feature for Google’s Gemini: how well the model uses your past Gemini conversations, Gmail, Google Search and YouTube activity to make its responses more relevant and helpful.

You design prompts from your own personal experience, then assess the personalised responses on dimensions such as grounding, integration and helpfulness.

Responsibilities

  • Design and run multi-turn conversational prompts (typically 1–5 turns) that require the AI to use your personal information and experiences.
  • Evaluate responses against the intent of your starting prompt, checking whether personalisation was applied appropriately.
  • Check grounding: claims about you must be supported by evidence, not flawed inferences or hallucinations.
  • Check integration: personal data should be woven in naturally, without robotic “overnarrating”.
  • Rank two model responses side by side to decide which is more helpful, easy to use and enjoyable.
  • Write clear, defensible rationales that reference the specific turns where issues or strengths occurred.
  • Extract and verify the model’s debug information to confirm chat summaries and data sources were used properly.
  • Delete evaluation conversations afterwards so they don’t affect your future chat history.

Requirements

  • Strong English reading and writing (English is the project’s focus language).
  • Willingness to use your primary personal Google account (not a testing account) and enable personal data sources for a genuine assessment.
  • Full-time availability in your local time zone (the team covers 24 hours globally).
  • Strong analytical judgment of nuanced, ambiguous AI responses, especially personalisation quality.
  • Experience designing creative multi-turn prompts from personal context.
  • Understanding of personalisation concepts: spotting incorrect personalisation, poor inferences and forced connections.
  • Close attention to detail when comparing side-by-side responses for naturalness and overnarrating.
  • Clear, concise, structured written rationales that cite specific turn numbers.
  • Constructive feedback, detailed annotation, and good communication and collaboration.
  • Self-motivated and able to work independently and remotely.
  • A desktop or laptop with a good internet connection.
  • A BS/BA or equivalent experience in a relevant field (e.g. policy, law, ethics, linguistics, journalism, computer science or a related analytical field).

Preferred

  • Experience in data annotation, AI quality evaluation, content moderation or a related role (strongly preferred).

Engagement and pay

  • Pay: not stated in the listing.
  • At least 4 hours per day and up to 40 hours per week, with 4 hours of overlap with PST.
  • Contractor, 3 months. Start date: not stated.
  • Process: shortlisted candidates get a job interest form; after a profile review, an assessment must be completed within 24 hours; based on the results, shortlisted candidates are contacted about pre-onboarding requirements.

Eligibility

  • Requires your own primary personal Google account with personal data sources enabled.
  • Requires full-time availability in your local time zone and 4 hours of overlap with PST.
  • Country or residence limits: not stated in the listing.