Member of Technical Staff (Evals)

Jarmin offers machine-learning engineering services to help companies build and deploy AI/ML systems.

Section II · Premium ProfileMembers only

7-day free trial · $25/mo · cancel anytime

Member of Technical Staff (Evals) · Jarmin

Job title: Member of Technical Staff (Evals)
Job location: San Francisco, CA, US
Job listing URL: https://www.ycombinator.com/companies/jarmin/jobs/K9ot5wF-member-of-technical-staff-evals
Job description: As a Member of Technical Staff on Evals, you'll own how we measure AI Employee performance on the job. **Areas you may work in:** * Evaluation systems: datasets, offline replay, scorers, and regression alerts * Feedback loops for models and agents * Autonomous failure analysis * Defining what good, degraded, and failed runs look like, and alerting on them **You may be a good fit if you:** * Have built evaluation or measurement systems, such as AI evals, experimentation, ranking, or search quality * Can turn ambiguous quality questions into concrete metrics and decisions * Have strong software engineering fundamentals and ship production systems * Are comfortable where there are few established patterns * Want to work in person in San Francisco **Even better:** * You've shipped LLM systems to production and learned where they break * You have strong data instincts and work well with researchers * You contribute to open source projects

The Saturday Briefing

New company profiles, funding moves, and who’s hiring across the market — every Saturday morning.

Where this role is based

San Francisco, CA

Loading map…

Jarmin headquarters

San Francisco, CA

Company size

2–10 employees

Founded

2025

Total raised

$500,000

Funding rounds

Investors

Industries