SECTION I · THE BRIEF
Brief #38141Updated 25 JUN 2026MOUNTAIN VIEW, CAGreenhouseKLEINER PERKINS
Employbl Company Profile

Machine Learning Engineer, LLM Evals & Observability

Glean is the work assistant with intuition. Search across all your company’s apps to find exactly what you need and discover the things you should know. 🔍 Search across your entire company to find highly relevant,…

Location
Mountain View, CA
Company size
200–500
Posted
1mo ago
Via
Greenhouse
Section II · Premium ProfileMembers only
  • 01Comp band & equity packageLocked
  • 02Seniority & experience requirementsLocked
  • 03Interview process & rubricLocked
  • 04Hiring manager & team contextLocked
  • 05Growth trajectory in this roleLocked
  • 06Offer & decision timelineLocked

7-day free trial · $25/mo · cancel anytime

Glean logo

Machine Learning Engineer, LLM Evals & Observability · Glean

View company profile
Job title
Machine Learning Engineer, LLM Evals & Observability
Job location
Mountain View, CA
Job description
About Glean:
 
Glean is the Work AI platform that helps everyone work smarter with AI. What began as the industry’s most advanced enterprise search has evolved into a full-scale Work AI ecosystem, powering intelligent Search, an AI Assistant, and scalable AI agents on one secure, open platform. With over 100 enterprise SaaS connectors, flexible LLM choice, and robust APIs, Glean gives organizations the infrastructure to govern, scale, and customize AI across their entire business - without vendor lock-in or costly implementation cycles.
 
At its core, Glean is redefining how enterprises find, use, and act on knowledge. Its Enterprise Graph and Personal Knowledge Graph map the relationships between people, content, and activity, delivering deeply personalized, context-aware responses for every employee. This foundation powers Glean’s agentic capabilities - AI agents that automate real work across teams by accessing the industry’s broadest range of data: enterprise and world, structured and unstructured, historical and real-time. The result: measurable business impact through faster onboarding, hours of productivity gained each week, and smarter, safer decisions at every level.
 
Recognized by Fast Company as one of the World’s Most Innovative Companies (Top 10, 2025), by CNBC’s Disruptor 50, Bloomberg’s AI Startups to Watch (2026), Forbes AI 50, and Gartner’s Tech Innovators in Agentic AI, Glean continues to accelerate its global impact. With customers across 50+ industries and 1,000+ employees in more than 25 countries, we’re helping the world’s largest organizations make every employee AI-fluent, and turning the superintelligent enterprise from concept into reality.
 
If you’re excited to shape how the world works, you’ll help build systems used daily across Microsoft Teams, Zoom, ServiceNow, Zendesk, GitHub, and many more - deeply embedded where people get things done. You’ll ship agentic capabilities on an open, extensible stack, with the craft and care required for enterprise trust, as we bring Work AI to every employee, in every company.
 
About the Role:

Building a great AI assistant is only half the battle – knowing whether it's actually great is the other half. Our team owns the measurement and quality layer that make Glean's Assistant and Agents reliably better over time: evaluation pipelines, quality eval-sets, LLM-powered judges, agent observability, and the tooling engineers use to understand what changed and why. It's a rare combination of infrastructure engineering, applied ML, and direct product impact. If you care deeply about quality and want to build the systems that make it measurable, this role is for you.

You will: 

  • Design and curate evaluation datasets – sampling strategies, query diversity, and golden sets that give reliable, representative coverage of real assistant behavior.
  • Build and maintain large-scale evaluation pipelines that measure assistant quality across thousands of real user queries.
  • Build LLM-powered judges that score metrics like correctness, completeness, and response quality, and align them against human judgment.
  • Evaluate new models and product changes before they ship – providing the quality signal that gates launches and prevents regressions.
  • Build observability infrastructure for AI agents: trace enrichment, data pipelines, and dashboards that make assistant behavior inspectable.
  • Close the loop between quality measurement and improvement using eval results, customer feedback, and techniques like automated prompt iteration to help drive concrete gains in assistant behavior.
  • Collaborate with engineers across the company to make evals a first-class part of how we ship.

About you:

  • 2+ years of software engineering experience with strong coding skills.
  • Strong backend fundamentals in Go and Python; comfortable with distributed data pipelines.
  • Experience working with LLM evaluation, reinforcement learning from human feedback, natural language processing, or other large systems involving machine learning.
  • Analytically rigorous – you think carefully about what offline metrics actually predict about real user experience.
  • Thrive in a customer-focused, tight-knit and cross-functional environment - being a team player and willing to take on whatever is most impactful for the company
  • You care about quality – not just in the systems you build, but in the product you're helping measure and improve.

Location: 

  • This role is hybrid (3-4 days a week in one of our SF Bay Area offices)

Compensation & Benefits:

The standard base salary range for this position is $200,000 - $300,000 annually. Compensation offered will be determined by factors such as location, level, job-related knowledge, skills, and experience. Certain roles may be eligible for variable compensation, equity, and benefits.

We offer a comprehensive benefits package including competitive compensation, Medical, Vision, and Dental coverage, generous time-off policy, and the opportunity to contribute to your 401k plan to support your long-term goals. When you join, you'll receive a home office improvement stipend, as well as an annual education and wellness stipends to support your growth and wellbeing. We foster a vibrant company culture through regular events, and provide healthy lunches daily to keep you fueled and focused.

We are a diverse bunch of people and we want to continue to attract and retain a diverse range of people into our organization. We're committed to an inclusive and diverse company. We do not discriminate based on gender, ethnicity, sexual orientation, religion, civil or family status, age, disability, or race.

#LI-HYBRID

 
AI-First Mindset at Glean:
 
At Glean, AI fluency is core to how we work and we're committed to ensuring every new hire feels confident integrating AI into their everyday work. As part of the interview process, you'll complete a brief AI-focused exercise or discussion so we can understand how you think about, design, and use AI to drive impact in your role. Feel free to reference any tools, platforms, or workflows you use today — prior Glean experience isn't required.
 
Global Data Privacy Notice for Job Candidates and Applicants:
 
Depending on your location, the General Data Protection Regulation (GDPR), California Consumer Privacy Act (CCPA), or other privacy laws may regulate the way we manage the data of job applicants. Our full notice outlining how data will be processed as part of the application procedure for applicable locations is available in our Privacy Policy. By submitting your application, you are agreeing to our use and processing of your data as required. US applicants and their applications are subject to arbitration of disputes as outlined in our Applicant Arbitration Agreement.

By clicking “Submit Application,” I confirm that I have read the Global Data Privacy Notice and the Applicant Arbitration Agreement, and I agree to the terms.  
View job listing ↗

Get the Saturday tech briefing

New company profiles, funding moves, and who’s hiring across the market — every Saturday morning.

Where this role is based

Mountain View, CA

Loading map…

Glean headquarters

Palo Alto, CA

Company size

200500 employees

Founded

2019

Total raised

$768,200,000

View company profile ↗

Funding rounds

  • Series F$150M
  • Series E$260M
  • Series D$203.2M
  • Series C$100M
  • Series B$40M
  • Series A$15M