- Posted 4 days ago
- Be among the first 30 applicants
Job Description
Software Engineer
Position: Software Engineer (SWE)
Type: Contractor
Compensation: $60–$120 per hour equivalent (paid per accepted task)
Location: Fully Remote (Worldwide)
Time Commitment: 15+ hours per week
About the Opportunity
We are seeking experienced Software Engineers with prior professional AI-training or model-evaluation experience to support projects focused on improving advanced AI systems.
In this role, you will apply your software engineering expertise to create challenging technical tasks, evaluate AI-generated code, identify implementation failures, and establish objective standards for correct solutions. This opportunity is intended for candidates who have already completed substantive AI-training, model-evaluation, or benchmarking work.
Key Responsibilities
- Create realistic and technically challenging software engineering tasks to assess advanced AI models.
- Develop clear task instructions, reproducible environments, reference solutions, tests, and evaluation criteria.
- Evaluate AI-generated code and technical deliverables for correctness, robustness, and completeness.
- Identify issues involving implementation, debugging, dependencies, performance, testing, and reproducibility.
- Assess software design decisions, code quality, and the reliability of proposed solutions.
- Establish objective standards for evaluating technical submissions.
- Provide structured feedback to support the improvement of AI coding capabilities.
Required Qualifications
- Prior paid professional experience training, evaluating, benchmarking, or improving AI systems.
- Strong practical proficiency in Python.
- Strong knowledge of software design, debugging, testing, version control, and dependency management.
- Ability to assess code quality, identify technical issues, and validate software solutions.
Preferred Qualifications
- Experience creating AI-training tasks, reference solutions, tests, graders, or benchmark problems.
- Experience reviewing technical submissions from other AI trainers.
- Codeforces rating of approximately 1900 or higher, or strong results in another programming competition.
- Familiarity with Linux, Docker, CI/CD, command-line tools, and sandboxed environments.
- Experience with long-horizon or agentic coding evaluations.
- Availability to begin immediately.
Compensation & Engagement
- Compensation: $60–$120 per hour equivalent, paid per accepted task.
- Engagement Type: Independent Contractor.
- Work Arrangement: Fully remote, worldwide.
- Time Commitment: 15+ hours per week.
- Schedule: Flexible.
Application Process
- Submit Your Application: Provide your application details, previous AI-training experience, and public technical evidence.
- AI Interview: Complete an interview that includes technical and integrity checks.
- Hiring Manager Review: Your application, experience, and technical qualifications will be evaluated.
- Onboarding: If selected, complete project onboarding and begin your first assigned task.
More Info
Key Skills
sandboxed environments
CI/CD
command-line tools
About Company
