Senior Software Engineer – LLM Evaluation
- Pay
- Pay not listed
- Where
- Remote
- Type
- Contract
- Eligibility
- Country eligibility not stated. Check with the provider before applying.
- Listing
- Last verified Oct 2, 2026
Apply opens the provider’s application page. You apply there yourself: we never submit applications, and opening the page doesn’t mean you’ve applied. Nobody can promise you’ll be hired.
Some Apply links are referral links. If you apply through them, the provider may pay us a commission. Referral disclosure
Overview
As a Software Engineering evaluator, you will work with researchers to create datasets used to train, benchmark and advance large language models. The work includes coding examples, precise solutions, code corrections and evaluation of AI-generated code.
Responsibilities
- Curate code examples, develop solutions and correct code for AI training in Python, JavaScript (including ReactJS), C/C++, Java, Rust and Go.
- Assess and improve AI-generated code for efficiency, scalability and reliability.
- Work across teams on AI coding solutions, assessing them against industry performance benchmarks, including enterprise-level use.
- Build agents that check code quality and find patterns in errors.
- Develop hypotheses about steps in the software-engineering cycle and evaluate model capabilities at each step: prototyping, architecture design, API design, production implementation, launch, experiments, monitoring and operation maintenance.
- Design mechanisms that automatically verify solutions to software-engineering tasks.
Requirements
- The listing names JavaScript and Python as required skills.
- Several years of software-engineering experience, including at least two years of continuous full-time experience at a top-tier product company. Examples given are Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal and IBM Research.
- Strong expertise developing full-stack applications and deploying scalable, production-grade software with modern languages and tools.
- Deep knowledge of software architecture, design, development, debugging and code-quality/review assessment.
- Excellent spoken and written communication for clear, structured evaluation explanations.
Engagement and pay
- Remote contractor engagement, with no medical benefits or paid leave.
- Flexible commitment from a minimum of 10 hours per week up to 40 hours per week, with partial PST overlap required.
- Stated duration: 1 month (starting next week; potential extensions based on performance and fit).
- Candidate compensation is not stated.
Eligibility
- The source says Remote and does not state eligible countries or a residence restriction.