Senior Software Engineer – LLM Evaluation | Remote

Senior Software Engineer Model Evaluation Work Snapshot Type Contract Commitment Approximately 1 week July 8 14 Minimum 10 hours

What You'll Be Doing

Familiarize yourself with assigned open-source codebases and their established engineering practices Review code changes technical discussions and related software artifacts produced by frontier coding systems Evaluate model-generated code for correctness quality maintainability and adherence to software engineering best practices Provide structured technically reasoned evaluations with clear written justification Collaborate with fellow technical reviewers to ensure consistency and quality across all evaluations

What We're Looking For

Strong experience in professional software engineering within production-grade codebases Strong experience in JavaScript/TypeScript (Node.js React) or Go (backend services distributed systems REST/gRPC APIs) or Python (backend development DevOps automation testing frameworks) Strong experience in code review with a focus on readability maintainability and software design Strong experience in Git and GitHub workflows including branching pull requests issues and code reviews Strong experience navigating and contributing to open-source repositories familiarity with SWE-Bench or SWE-Bench Pro is a plus Strong English written communication skills with the ability to articulate clear technical reasoning

Application Process

Submit your application through the Easy Apply button. Each application will be reviewed against the role requirements. Candidates who meet the requirements will receive an email with the next steps. Follow the instructions provided in the email to complete the remainder of the application process.