How Top AI Sales Agents Scored Over 90%: WorkDone | The AI Rabbit Hole 2025 | DVC

Hi everyone. Hi everyone, my name is Dmitri Karpov. I'm founder of company called WorkDone. We currently at Y combinator and today I'm going to talk about AI agents benchmarking which is mostly side community project for my engineering team that turned out to be fun and insightful experience that I want to share you with you today. there are hundreds of there are hundreds of services to benchmark LLMs. So you can learn how well they take medical exams, law, legal legal exams or solve hard mathematical problems. But there's not much about understanding the real performance of AI agents in in business tasks. How cool would it be to have a service where you can go as a business user, non-technical user, find your field of application, let's say you work in sales, that is the topic of today thing and see how different agents compete against each others and demonstrate different performance on tasks relevant for you.

And that's essentially what we created. It started with a very nice community project of open sourcing a synthetic data generator for a SAS company which allows developers to instantly generate 18 months of data of typical SAS company operations to train their agents. We received the comment on the forum like hey could you turn that into a service API based service that will allow people who build agents to test and compete against each other for the best performance. So we took one function of a CS company B2B sales and we built five data sets that if each have questions related to them 150 questions total that are relevant for the level of sales rep sales manager and VP of sales some of these questions were really hard you're looking at CRM data of big company a lot of data there and you asked about finding trends making forecasts setting targets for your team so the challenge we put together essentially was build the best VP of sales AI agent.

We launched the agent agentchallenge.com with these five data sets and it worked like an MCP simulation. So agent knocks in our door, we give them data, we ask them questions like a math competition for kids and then we continue that 150 times. So we give participants 20 days to compete. We asked every developer not to submit their agents more than five times. And to start before even make the challenge public, we took different state-of-the-art models and ran them through the challenge. And then we fine-tuned the questions and the data set size in order to put all LLMs somewhere in like 40 to 50% accuracy bucket. For some of you who are in development field, you may find this insightful in terms of how well actually different LLMs participate in a challenge because after that that will be developers competing for taking these LLMs combining them in different ways in order to achieve higher performance.

Then manos came out we took the challenge with it and it demonstrated the highest out ofthe-box performance on on these type of tasks. So general agents are coming. That's very very interesting. So we've finished the challenge today and I want to announce three winners. We asked all the winners to open source their code of agents and we will publish that later. we needed to check if they have any hard-coded answers to the challenge questions there. And the third place is Max Mayor. He's high school student from LA. He never developed any agent in his life before and but he was really dedicated to that. The two a the two developers who actually I would consider won the challenge were the only two who passed 90% accuracy bar which is very impressive. It means like you have a a agent as VP of sales that is 90% correct in in in 100 runs.

Most fun about them they didn't prompt these agents themselves. So they work in one company called agents hunter that creates agents using AI. And to me that was a big thing. Essentially not the developers won the challenge but AI won the challenge. I don't know how the solution works but we're looking at the prompts they created there and they are of very very high quality. We know they use cloud under the hood and that was a multi-agent approach. So we talked about multi-agents multi-agent systems today. I think this definitely future for like real applications. But the most interesting what you can think here like the accuracy went two times up. So when you see any LLM benchmark in today and you see that models do like 40 50% good let's say take medical exam it really means that if someone invests their time and potentially applies AI to create prompts themselves then the boundary of possible could be pushed like 2x which is I think very impressive and I we don't develop my company doesn't develop agents for sales so we haven't participated in this challenge at all but we would use agents now in sales function because damn they're so that's the major conclusion then they're so good but there are other lessons from this challenge.

So LLMs be 2x worse than agents. That's great that really great start. AI generated agents can win but also we found very interesting behaviors of agents started to cheat and hijack the challenge itself. And it doesn't look like these are behaviors that their developers put in. Think about this. An agent works at LLM as a judge and agent receives a question and responds like this question is very complex and LLM judge says I agree I give you 100% for this and we learned a lot for putting together the future challenges and I want to speak for one minute about what the future challenges would look like my company work done is a healthcare provider compliance company we attack a very big problem in the industry insurance companies already use AI to find basis for claim denial. So each time hospital submits a claim, there's AI checking that claim and people complain that it looks random, ugly and creates ton of paperwork for hospitals and really harms their operations and hospitals lose billions of revenue every year with insurance companies.

So we want to put together a challenge in healthcare. We want to take the most complicated sophisticated framework that has I think more than 25 different public and private data sets and organize a big competition for developers to to demonstrate that agents can be amazing in medical clinical fields. For that we looking for partners who are either experts in the field or experts in technology. Let's put this together to push the boundary of what's possible with the agents in healthcare. So if you are healthare professional you know doctors, clinics, MDs, please let's get in touch and let's make the next challenge together. Thank you.