AI Operations Analyst

Calltree AI, founded in 2025 and based in San Francisco, is a SaaS company that leverages generative AI to optimize contact center workflows. The company's AI agent analyzes millions of minutes of contact center…

Location: San Francisco, CA
Company size: 2–10
Posted: 3d ago
Via: Yc

Section II · Premium ProfileMembers only

01Comp band & equity packageLocked
02Seniority & experience requirementsLocked
03Interview process & rubricLocked
04Hiring manager & team contextLocked
05Growth trajectory in this roleLocked
06Offer & decision timelineLocked

Start free trial Sign in

7-day free trial · $25/mo · cancel anytime

AI Operations Analyst · Calltree AI

View company profile

Job title: AI Operations Analyst
Job location: San Francisco, CA, US
Job listing URL: https://www.ycombinator.com/companies/calltree/jobs/qNST1yz-ai-operations-analyst
Job description: **About Calltree:** We're a YC W25 company with multiple enterprise customers, including public retail brands you'd recognize. We're building AI that actually works for complex, real-world use cases. For example, our AI handles 30+ minute technical troubleshooting calls—the kind everyone else avoids. **The Role:** You'll be the human supervisor behind our AI agents, working with live customer interactions in real-time. You'll monitor how AI performs, ensure quality, and when needed, jump in to rescue conversations and ensure great customer outcomes. You'll also work directly with engineering to make the AI smarter every day. **What You'll Do:** * Monitor live AI-customer conversations and ensure quality standards * Make real-time judgment calls on when to intervene * Step in as the escalation point when customers need a human, de-escalating frustrated customers and resolving complex issues * Identify patterns in AI performance and document what's working and what's not * QA new AI experiences before release—interact with them as customers would and flag issues * Work with engineering to improve the AI based on what you observe * Build the playbooks and processes for this function as we scale **What Success Looks Like:** * Measurable improvement in AI performance metrics AND customer satisfaction * Good judgment on when to intervene vs. let the AI continue * Catching issues before they hit production * Building institutional knowledge about what makes conversations succeed or fail **What We're Looking For:** * Recent college graduate (any field) * Comfortable being on the phone with customers, including frustrated ones * Someone who's excited about AI technology AND genuinely enjoys helping people * Strong judgment and decision-making skills * Ability to work one weekend day **Honest Expectations:** This role is a balance of two things: operational monitoring work (watching, reading, documenting) and live customer interaction. Some days will feel repetitive. But you're also accountable for real customer outcomes—when you jump in, you're the person who saves the experience. If you love technology but never want to talk to customers, this isn't the role. If you want both—mastering AI systems AND helping real people—you'll thrive here. **Career Growth:** This isn't just a job—it's a launchpad. Top performers have clear paths to become AI Team Leads, move into Product roles, or specialize in AI training and development. You'll gain hands-on experience with cutting-edge AI technology that will set you apart in your career. **To Apply:** Send your resume with a short video (2-3 minutes). For the video: pick an appliance or product you have in your house, then roleplay a customer service scenario where you're helping a frustrated customer troubleshoot a problem with it.

View job listing ↗

The Saturday Briefing

Get the Saturday tech briefing

New company profiles, funding moves, and who’s hiring across the market — every Saturday morning.

Where this role is based

San Francisco, CA

Loading map…

Calltree AI headquarters

San Francisco, CA

Company size

2–10 employees

Founded

2025

Total raised

$500,000

View company profile ↗

Funding rounds

Pre Seed$500KMar 12, 2025

Investors

Y Combinator

Industries

Software Companies Software As A Service Companies