AI Evals Office Hours

Open office hours where non-profits bring the practical AI evaluation problems they are actually hitting, and we work through them live.

Resources

• Slides

TranscriptCopy transcript

0:00and skills and many different tools.0:03So, you know, we want to make sure that we have a very clear baseline0:09and a very, very clear way to evaluate before we start layering0:13and all these different tools.0:15So that's where we are. Thank you.0:18Awesome. Yeah. Thanks a lot, Joel.0:21Ayushi?0:22I'd love to go next. Hi, I'm Ayushi.0:25I work at a non-profit based out of India called the Antara Foundation.0:31The Antara Foundation specializes in, it's a decade-old non-profit0:37specializing in improving public health outcomes for maternal and child health outcomes.0:45We have started a digital initiative, which is fairly new in the organization.0:52And what I'm struggling right now is how to sort of balance the ever-increasing knowledge base,1:02which is public health guidelines, with the speed or with the pace1:08with which we want to move a newly established team.1:12Meaning some of the questions I keep thinking about every day is,1:15do I need to run evals after every time I add a couple of guidelines1:21or a couple of PDF guidelines in the knowledge base?1:25Do I need to change the eval design?1:27How do I go beyond cosine score and correctness score?1:31If there's a third tangent of languages, how do I address those as well?1:38And all of this keeping in mind that the product is essentially a RAD-based AI chatbot1:44for community health workers who might not be super familiar with what the bot is saying.1:53So 30% of the users know what they have to do.1:58And chatbot essentially acts as a verifier,2:01while the remaining 70% are not as fluent in their day-to-day jobs.2:06And what keeps me up in the night is whether a wrong answer, let's say, or not,2:11or a hallucinated answer by AI,2:14I hope they don't take that as a ground truth and take that for a record2:19and make wrong decisions on the field because they're as informed as the AI.2:24So these are some of the questions that I'm struggling with,2:27and I'd love to sort of learn from the group and learn from you as well.2:33Awesome. Thanks a lot, Ayushi.2:36Great. And thank you, Ahalya, as well, for sharing your background.2:43I think we'll sort of start with the session now.2:45So just because I saw a bunch of common themes, I guess I said I don't have anything to present,2:54but maybe just a high-level overview might be helpful for us to ground us in the same terminology.3:04So, yeah, let me just present my screen. One second.3:30Yes. Can you see my screen now?3:31Yes, we can see.3:40Yeah. Okay. Cool. So in general, when we say agent, we might mean multiple things.3:48It could mean like a single LLM feature, like we said, hey, translate this text3:53or summarize this paragraph or this document.3:56It could mean like a workflow where we might chain multiple models together.4:01It could be chaining like a speech-to-text model with an LLM followed by a TTS model.4:07It could be chaining one LLM with another LLM.4:10It could be running multiple LLMs in parallel and combining their outputs,4:14but it's a fixed flow, like you have fixed this flow.4:18Or it could be like an agent, for example, which is deciding its trajectory.4:22You run an LLM, it decides what to do next, and it just keeps going on until it feels4:27that it is done. And it could be like multiple agents also coordinating with each other.4:32So typically, people nowadays call everything an LLM.4:36So this is the simplest case.4:37It could be like, as many of you have mentioned, you have a knowledge base.4:42So instead of just a simple call, the query first goes to a knowledge base.4:46It finds the relevant documents.4:48Then those documents are sent to the LLM, which responds back.4:52It is also possible that the LLM might call tools, for example.4:56Suppose if you want to respond based on a query in your database, for example.5:02If suppose you want to respond based on your email,5:05someone might have asked, hey, what did this person say today?5:08So the agent might choose to go and read your email.5:11Or you might say, summarize all the discussions in my company chat today.5:15So it might go through your Slack and then sort of summarize that.5:19Or if you are asking, who is the president today?5:22Models typically don't have up-to-date knowledge.5:25They have been trained up till a particular date.5:27So these models need to then go and search the web.5:30So even searching the web is a tool.5:34Is everyone with me so far?5:38Yeah.5:39Yes.5:40So generally, all of these can be called agents.5:44But if you look at the final output of whatever we call agent, whatever it produces,5:51it can be broken down into one of these two categories.5:54It either decides to give a text reply or it decides to call a tool.6:01Even if you sort of think of retrieving from your knowledge base,6:05that is also kind of like a tool,6:07where the retriever is a tool that the agent is choosing to call6:12before it generates a response.6:15So when you want to evaluate your agent,6:17it can help you to think that at the end,6:21you just want to evaluate either its reply or the tool that it is calling.6:28That's all the evaluation eventually boils down to.6:35Now, I can sort of quickly walk through like a use case6:40of how we do evaluation for one agent.6:44And it might help you understand like how this sort of happens.6:48And because we work with evaluating for many non-profits,6:54I personally built like a tool for non-profits6:58to do this evaluation with domain experts.7:00Typically, we don't have technical capacity in the team.7:04And even in the form, one person had said that there is a struggle between,7:09like we have limited engineering bandwidth.7:11The same people who are building the product, building the AI engines,7:15asking them to evaluate takes up their time.7:18And often that time is not available.7:21And so the solution has to be that the domain experts have to be involved in it.7:27But there have been hardly any tools out there7:29which enable the domain expert to be as involved as well.7:35So we'll take a very simple use case.7:38And then we sort of walk through how we do evaluation here quickly.7:42So we have been working on a voice agent that fills form.7:47So forms are everywhere in the public health system.7:51It's filled of full of forms, right?7:54And what happens is that India has a massively digitally less literate population.8:00So they can't read these forms.8:02They can't write the answers back.8:04So someone has to always talk to them.8:06And that job comes down to health workers.8:08Now, these health workers are already very limited in capacity.8:11And so asking them to speak to one person at a time8:16is taking up their massive, their limited bandwidth.8:20And it also is capping how many people we can reach at the same time, right?8:25Because every person is speaking to the health worker.8:27So the idea was instead of the health worker speaking to the end user,8:32can we have a voice agent that directly speaks to them?8:35So very quickly, like, can you hear this?8:40Is it audible to you?8:42Yes.8:43Yes.8:44Yes.8:46So, okay, I don't know, it just.9:04So it asked the name first, and then the name was filled here.9:13Then it asked the district.9:17Okay.9:22So similarly, it sort of, like, for example, here I am.9:25I am just saying, oh, I don't know.9:31So similarly, it sort of goes on for the whole form,9:33where the agent is talking to the user.9:35And based on their response, it's filling a structured form.9:39Now, this problem, like, we have like a particular architecture9:43for how we are sort of solving it.9:45And we sort of have like a speech to text model.9:50And we have different LLMs, which are being used.9:53One LLM is extracting the value from whatever user has said.9:58And one LLM is deciding what to say to the user.10:03Now, there are many issues that can happen here.10:05When the user is in a background, is in an environment10:09where there's a lot of background noise, the speech to text can be wrong.10:12There could be multiple people speaking in the background.10:14And it might be hard to know which speaker's voice is getting captured.10:19And typically, these models struggle with low resource languages,10:23with transcription in general.10:25Similarly, the LLM that is extracting the value,10:27it might extract the incorrect value.10:31If the user has said an incomplete value, it might accept it instead of rejecting it.10:35If the user sort of wants to skip a mandatory question,10:40the LLM might let it skip.10:42Instead of saying, no, you cannot skip it.10:44And similarly, the agent that is replying, it might end up messing up.10:49Like, for example, if the agent is supposed to reply with one line,10:53it might end up replying with like six lines.10:55If the user has asked in Hindi, a particular language,10:58the agent might end up responding in English, which is not desirable.11:03So each of these components can fail on their own.11:07And each of these sort of propagate into each other.11:10If the speech-to-text fails automatically, the LLMs downstream fail as well.11:14Right.11:15Is everyone with me so far?11:19Is there any doubts?11:20Feel free to ask me.11:23Yeah. Okay.11:24So that is why we need to test it.11:27Right.11:27Now, let's see how we can test it.11:30So we've been building this tool called Calibrate,11:35which is specifically for evaluating agents within non-profits.11:39Okay.11:40Because non-profits have certain constraints, we don't have like a lot of tech bandwidth.11:46We typically need to self-host these tools inside because our data is sensitive.11:51We typically can't pay the massive price that many of these tools online11:56ask us to pay per seat pricing.11:59And we don't lock into any vendor.12:01Right.12:01Which often a lot of tools also do.12:04So let's see how this sort of works.12:06Earlier, I said that we wanted to, we want to test either the next reply here.12:15Either we'll test the reply or we will test the tool that it calls.12:21Okay.12:21Now, I have two of my agents here.12:24As I said, we have the two LLMs that we want to test.12:27Now, if I just look at the first LLM.12:30Okay.12:31This is like a test case that I have made.12:34What does this mean?12:35This means that given this conversation that has happened so far,12:41what should the agent respond with?12:45So here, what I am testing is suppose the agent got this as input,12:50then these are my evaluators.12:53So here I am checking all these different dimensions of evaluation.12:58For example, Ayushi had mentioned that,13:01what if I just don't want to look at correctness or like accuracy?13:05I want to look at more stuff.13:07So here you can look at different dimensions.13:10For example, here I am checking, hey, is the response adhering to Hindi?13:15And it is using a particular script.13:17Is the response concise?13:20Is the response correct?13:22Like what does correctness mean?13:23So for example, it could be correct, but it could reply in English.13:27Right.13:27So making it incorrect.13:29And if I use just a single criteria to box everything together,13:33then I don't know what was wrong.13:35Did it ask the wrong question or did it reply in a different language?13:41Was it like too long?13:42I don't know.13:43So think of these as like different dimensions of evaluating the agent on.13:49Right.13:50Is everyone with me so far?13:51Like every LLM will get the conversation history as input13:55and then it has to produce either a reply or a tool call.13:59Here I am evaluating the reply.14:03Is that super clear to everyone?14:04Please let me know if you have any doubts.14:10I'm happy to explain again.14:22OK.14:23OK.14:24So we have this input.14:26We have the dimensions.14:27And then we can run a single test.14:30Right.14:31Now what this will do?14:32These it will pass the input to my LLM.14:35The LLM will generate a response and then that response will go to different LLM judges.14:41These are these are like AI models which are evaluating the response itself.14:46So, for example, here my agent replied with this.14:52And now there are separate evaluators which evaluated each dimension.14:57So here we can see that it passed every single dimension.15:02Right.15:02But it is possible that if my agent was not correct, it might have failed one of these15:06dimensions and then I would know, OK, it is passing on all these six dimensions,15:11but it is not passing the seventh dimension.15:14Right.15:14Now this is just one test case, but you can see I have made 500 tests.15:19Why 500?15:21This is meant to capture the diversity of inputs that the LLM might get.15:27I just showed you one edge case.15:29Right.15:30And a lot of people told, hey, how do we monitor this in production?15:33That was a common question across people.15:36Right.15:37As and often you make changes and once you make a change, you don't know if you broke15:42something else.15:43Right.15:43And this is the idea of unit tests from software development.15:47Typically in software development, we would write unit tests to test our code.15:51Right.15:52So that before we push out anything, we make sure that we didn't break anything else.15:57This is the unit test equivalent for agents and specifically for their individual components.16:04Now, all these 500 tests can be thought of as like 500 unit tests, which capture the16:10diversity of inputs that I might receive.16:14Like you can see different types of conversation histories, different types of user inputs,16:19user refusing to respond, user complying with the response.16:23And this helps me test what should my agent do in every single scenario.16:29Right.16:30And then when I run these 500 tests, for example, I see that it is passing for 98.6 percent16:37of cases.16:38Right.16:38And if I can look at these particular failure modes that, OK, in this case, this particular16:45dimension failed.16:46Right.16:47And I can then go back and inspect, look at my prompt and understand why did it produce16:52this way.16:53But then whenever I release a fix, I again run all of my tests to make sure that this17:00number has not deteriorated.17:03Right.17:04And that is the way we can and we also compare it with your like per request latency, the17:10cost that it takes, the cost per request and the number of tokens per request.17:15And this way you can check that, hey, when I push a release, has my pass rate improved?17:22Has it reduced?17:23Right.17:23So that I can deploy with confidence.17:26And beyond that, we again don't care about the model.17:29Right.17:29We want the best model.17:31So what we let you do, like what the tool helps you do is just you can select whatever17:36models you want to compare and it can tell you which model performs the best for your17:40use case.17:41So if a new model came out today, you just spin it up, spin up your test cases and you17:47get the best model for your use case.17:51I'll stop here, but I'll sort of have, I'll open it up for more questions.17:56But this is one way that I wanted to show how we run evals.18:02Right.18:02And the good part of this is none of this requires the domain, none of this requires18:07writing code.18:08Right.18:09So we hand these off to the domain expert once we've built the agent and the domain18:14experts can come and create more test cases, for example.18:18Right.18:18Like, OK, I can imagine this is one scenario.18:21The user says, hi.18:23The agent says, hello, what is your name?18:25And the user says, my name is Aman.18:29And the test is that, OK, it asks the next question.18:34Right.18:35And I'm adding a criteria that, OK, ask the next question.18:40Don't repeat the answer, for example.18:44Right.18:45And and here I have my test created and I run my test.18:49So it's as simple as that.18:51Right.18:52The idea of building with this was that we hand it off to the domain experts and they18:58can take ownership of these tests.19:00Right.19:00And they can come back to us after looking at the mistakes with targeted specific feedback.19:07A similar thing happens for tool calls.19:10But before going there, because a lot of you asked about answer quality, I'm curious to19:17know what you think, what are your doubts?19:19And this is one way that we do it.19:21But yeah, happy to hear your thoughts on this.19:24And if you have your questions, please feel free to ask, what do you see as the limitations?19:28Please go ahead, Joel.19:31Yes, thanks for sharing the tool.19:34I remember I can't remember the actual date, though.19:37I think it was this year.19:38Right.19:39You there was a webinar earlier this year where you demonstrated the tool.19:43Can you can you shared like any of the updates since since that since that webinar?19:52And since since that demo?19:55Yes, yes, we have been.19:58So a common question was that.20:00People asked, hey, OK, we are using these LLM judges,20:03but how can we rely on them?20:06Because the LLM judges themselves might be wrong.20:10And so what we have done is we have also added support now20:15for calibrating your LLM judges.20:18How that works is essentially you can, yeah,20:24you bring in your actual live traces here,20:30and you sort of have annotators, human annotators20:34review the samples and compare that20:38with how your annotation, how your evaluator is working.20:42So for example, here I can see that I20:45can compute the alignment of this particular LLM judge.20:49And for this sample, my evaluator20:51said that, hey, this is wrong.20:54But the human said that this is correct.20:56So in this place, the LLM judge disagreed with the evaluator.21:02And so we can look at all the disagreements.21:04We can identify, hey, what is the problem?21:06We can export these results out, and we21:09can analyze them separately.21:10And then we can come back and update the prompt.21:13At the end, my evaluator has just multiple prompts.21:18It has different models.21:19And I can update the prompt based21:21on whatever I have learned.21:23And then I can check if the alignment increased.21:26So in this case, you can see that the previous version21:28had 33% alignment only.21:30But this version has 100% alignment now21:33with this small set of 43 samples.21:35And so over time, as you keep bringing in more samples,21:38you can track how the agreement of your LLM judge grows.21:42Once your agreement is close to 95% or 100%,21:45then you can start trusting the LLM judge judgments.21:49And that can help you filter out which responses do you really21:53need to inspect.21:54Because it is not possible to inspect21:55every single production response for where it is failing.21:59But once you have a reliable LLM judge, then you can do that.22:03Another mistake that we've often found people22:06do when they do LLM judgments is they often22:09have a Google Sheet, which is shared22:11with multiple annotators.22:14So one annotator is looking at the response22:17of the other annotator.22:18And so it's not like a blind, unbiased response.22:21So we let you create separate, independent annotation links.22:26And each of these annotation links22:28can be sent to individual annotators.22:31And so they just see their own interface.22:33They don't see what another person has done.22:36And they can look at this.22:38They can add their judgments very, very simply,22:41go through this, add any reasoning, add any comments.22:44And the feedback is done.22:46And that then automatically compares with your LLM judge22:50score and gives you an overall alignment.22:53How aligned is your LLM judge?22:55And if you have multiple annotators,22:57there's a reason that why we specifically22:59added annotator agreement.23:01Because a lot of people are often23:03not using multiple annotators.23:05For many use cases which are subjective,23:08we humans all vary.23:09What I think might be right, you may not agree with it.23:13But we often don't even capture that.23:15Because we are just using one annotator.23:18So we make this a default here so23:19that we ask people that, hey, you want multiple annotators.23:26And here we can capture those multiple annotator agreement23:29to see if there is consistency between your human annotators23:32to begin with.23:33Because if they are not aligned, then23:35we can't hope to achieve LLM judge alignment.23:39So this has been one big feature that we've added.23:42Beyond that, we are working on a lot of people said that,23:46can we use this with Claude Code directly?23:50It is very confusing.23:51It's hard to use the UI.23:53There are too many elements.23:54And people are now used to Claude Code.23:56So a big feature update that we are working on23:58is letting you just use Claude Code and interact with that,24:03talk to it, give your CSV files.24:05And it will run the evals on calibrate,24:07give you the results back, analyze it for you.24:10And you don't ever have to leave your agent interface.24:15Yeah.24:18Does that answer your question, Lloyd?24:19Yeah, thanks for those updates.24:21And I was fully aware of this tool.24:24But I hadn't revisited it in a while.24:26So thank you.24:27Thank you for the time that you're putting into this.24:29Awesome.24:29And any feedback that you have?24:31Because you also asked how to monitor answer quality24:33in production.24:34Does this look like something that24:36is heading in the right direction for what24:37you want as well?24:40Yes, I believe so.24:41Yeah, I think that, yeah, I guess24:45I'm not clear on how the tool, how24:48you would evaluate in production versus in development.24:53I mean, this is just evaluating the development, right,24:58before you would push it to production.25:00Yes.25:01So that's the next feature that we are working on25:03is adding support for tracing.25:04So once we add that, like, for example,25:07we have your agents here.25:08Like, the way you see all your tests here,25:10you will just see traces here.25:12So all your traces will show up.25:13You just add a code snippet.25:15All your traces will show up.25:16We will run the LLM judge on those traces.25:19And so it will identify which ones are wrong.25:22And then those can be converted to tests.25:25So that way, your unit test suite25:27keeps increasing over and over.25:29As you keep finding new issues, your set of tests25:32keep increasing.25:33So traces is the next step that will enable that on production.25:37Yes.25:37OK, great.25:38Thank you so much.25:39Awesome, yeah.25:41Yes, Jonathan.25:43Good morning, or good afternoon, or good evening, everyone.25:46This is my first, I guess, my first sort of foray25:50into what you're doing here.25:52It's really interesting, I think,25:53both from your methodological insights as well as25:56the tooling.25:58One question I have, just in terms of this piece,26:01is the LLMs themselves are not static.26:04So how does that influence the way that one goes about testing?26:09So you just showed the chart where26:11things are trending in the right direction.26:12Everything looks good.26:14How frequently, and how do you go about sort of retesting?26:18Because what your LLM gives you today,26:20it might give you a different response next week.26:23Yeah, I think it depends on where you are.26:28If your model is severely bad, if the model26:31is performing severely bad, it may not26:34be entirely because of the model's limitations.26:37It could be because your prompt is wrong,26:39or you may not be feeding it the right knowledge base.26:42So I often just start with the best model out there26:47and worry about cost, latency, trade-offs later on.26:50Because if the best model itself can't serve your performance,26:54serve your needs, or get the right accuracy,26:56there is no hope with the smaller models.26:58Of course, we can get into fine-tuning a model specifically27:01for your use case.27:02But fine-tuning essentially means,27:05for those who may not know, you take27:09a model that exists out there and you train it27:11on your data set.27:12But it is often much more trickier to do than it looks.27:15And it should really be resorted to as the last solution27:22after you've exhausted all the options of using27:25the existing model.27:27So if you use the best model out there,27:29and if your use case feels like something27:32that you think that you can do easily,27:36or it's not rocket science, or it's27:38more straightforward, which a lot of the applications27:42often are, they are reasonably straightforward.27:44And you think that the agent should be able to do it.27:49And if it is not able to do even decently,27:54then very likely that the problem is with your prompts,27:57or not giving it the right tools,27:59or not feeding in the right knowledge.28:02And I would iterate on that first28:04before trying to do any model changing,28:07or experimenting with different models.28:09But one thing that we-28:10Yeah.28:11I'm sorry, I'm actually concerned28:13about the opposite, where everything28:15passes testing, what is the risk that a week or a month later,28:20the performance is going to degrade28:22because the LLM has changed?28:27If you don't update your LLM, then maybe I'm28:31not able to understand.28:33When you say LLM has degraded, is it28:35because the underlying data or the query28:37that you are receiving has changed?28:40Because if you have a model deployed,28:42can you explain what you mean by the model degrading?28:45Not necessarily the model degrading,28:47but the performance degrading because the model itself28:51has changed.28:51So I might ask a question of Claude today28:54and get one answer that seems pretty good.28:56I might ask the same question next week28:58and get a different answer that's not quite so good.29:01I mean, I find in my limited efforts, which involves29:04essentially prototyping, Claude, even though I've29:07got rigid rules and I feed it the same context all the time,29:11it can still be forgetful.29:13It will miss things.29:14It will answer things differently.29:16Yes.29:17I will occasionally get hallucinations.29:19Yes, I agree.29:20Makes sense.29:20Got it.29:21So here, I think that's what I was telling Joel about as well,29:25that we are soon going to add support for traces29:28where you can see the production queries here itself.29:32And then we have the LLM judges, which29:34are doing the judgment, which are judging29:36the quality of the response.29:38And they will be running on these traces regularly.29:42And so you will get a live score of how29:44your model is performing across different dimensions.29:47Now, it is very likely that your LLM judge might pick up29:51that, hey, the response quality is degrading29:54because you have aligned the judge to human preferences.29:58So the judge might pick up that the quality score has reduced30:03now, and that can be one signal.30:05But what we recommend is that the judge30:08may be aligned also on a previous set, which30:11is an ideal set of examples.30:13So what we recommend is that you always, at a weekly basis,30:18use some of those traces and create human annotation jobs.30:22So every week, maybe 20 or 30 samples is reviewed by a human.30:27And then you compare how the LLM judge reviewed them.30:31If they continue to match, then you30:33can use that as a proxy that your LLM judge is still30:37reliable.30:38And so if it is saying that everything is fine,30:40everything might be fine, unless, of course,30:42the user reports are an error.30:44And then you go back and see, hey,30:45why did the LLM judge not pick up?30:47But regular human alignment checks with the LLM judge30:52builds in the trust in the LLM judge,30:55which is then supposed to catch the quality drops30:59in production.31:00It may be because the type of inputs31:03that we started to receive have completely changed.31:06But that's the process that we are31:08trying to build here, that continuous loop.31:12Yeah, does that make sense, Jonathan?31:14Yes, absolutely.31:15That's really, really interesting.31:17Awesome, yeah.31:20Anyone else, please feel free to just unmute and speak.31:24And I'll also go through the questions31:26that we have received on the form,31:27and we'll try to address them.31:29But if you don't have any other questions,31:31I'm happy to go through the questions now.31:39OK, let's go.31:43So these were our key questions.32:00OK, so a bunch of you said that we are not32:03evaluating in a thorough manner.32:06We are just doing spot checks.32:07A lot of it is reactive.32:10From what we have discussed so far,32:12do you have any thoughts, any reflections32:16that you would like to share?32:17And please point out any flaws in whatever32:21we have said so far that doesn't apply to your use case as well.32:24But this is just sharing a vision of what we have in mind32:27so that you don't have to rely on just spot checks.32:31So yeah, please feel free to unmute and speak up.32:37OK, Ahalya said that your major pain point32:40is writing golden data sets.32:42Can you help me with the strategy?32:44Yes, OK.32:44So the way we think of, at least I think of it,32:47is layering it up in different ways.32:49I typically use LLMs for creating a lot of my golden32:54data sets wherever possible.32:57And I start off with just the ideal use cases.33:01So a lot of times when we build an agent,33:04even the agent doesn't even work on the ideal inputs33:08that we can assume.33:10So I always start off with, what is the expected ideal input33:16to my system?33:18Thanks, Joel.33:19What is the expected ideal input to my system?33:23And maybe let's take the particular example33:26of this form agent to illustrate this for you.33:32Now, the ideal input is that the person says their name33:35properly, they say their district name,33:37they say clearly that they are pregnant,33:40they give the right birth date, they33:44give the right phone number.33:45But it is possible that when someone asked the district,33:48the user said, I don't know the district.33:51And so the agent needs to handle that.33:53Or suppose if my district was a fixed set of values33:56like Pune, Hyderabad, but the person said Mumbai,34:01which is not in the list of dropdown.34:03So it's an invalid value.34:04How does my agent handle that?34:07Now, when I've asked for their birth date,34:10their child's birth date, the person may not include the year.34:14They just say the day and the month.34:17And depending on my use case, that may or may not be correct.34:20So these are different edge cases.34:21But just to begin with, I will assume ideal inputs.34:27OK.34:30So, Ahalya, please feel free to, OK,34:33I think you have some speaker issues.34:35But please feel free to give your feedback.34:37But that is how I sort of begin with.34:39I map out all the possible variations that34:42are possible for my agent.34:44In this particular use case, it's a form.34:47And the variations can happen at a field level.34:49It could be variations in the district,34:51variations in asking for the birth date of my child,34:56or asking for the phone number.34:57The phone number can be set in many ways,35:00333, 333, 335, 3355.35:05So there could be many variations35:06in the way it is set.35:08So I try to map out all the variations first.35:11Then I look at what are the easiest ideal cases.35:16And I try to use an LLM generator35:19to generate those edge cases for me.35:22But often, these LLM generators may not35:25often produce the right output right away.35:28So I also have a separate LLM verifier.35:31Because verifying is easier than generating.35:35Because during the generation,35:36it might be looking at multiple things.35:38And then it might miss something35:40or produce something incorrect.35:43So we have an LLM generator and an LLM verifier.35:47The LLM generator keeps producing a new sample.35:51And the LLM verifier keeps verifying35:53if the sample is adhering to my requirements.35:56And if it does not,35:57it feeds the error back to the LLM generator.36:00And this way, after some iteration,36:02I get a data set up.36:04And I would try to refine this loop36:06on a smaller data set on the ideal samples first,36:09and then keep bringing in additional complexity.36:13But often, we start off with ideal inputs,36:16is my agent working?36:18If it is not, then try to fix it.36:20But if it is, then we start bringing in36:22the next level of difficulty.36:24Because often, these difficulty levels can be ranked36:27from no difficulty to the highest level of difficulty.36:30So I would think of building your data set36:32in this staggered manner,36:33from easiest to the hardest.36:35And only build the level of difficulty36:39where the agent is failing right now.36:41Don't aim for a higher level of difficulty.36:44And so this way, your data set can be built incrementally,36:47not all at once.36:53Awesome.36:58Okay.37:01Anyone else, please feel free.37:03I think we can go through some other questions.37:07So this question is,37:08funding the necessary development and testing costs37:12to get the AI product ready for deployment.37:15It is easier to get research funds37:17for evaluating the impacts of our established product.37:20But the product that we are building,37:22designed on the back of research,37:24reveals the need clearly,37:25will require substantial user testing37:27and is not common in our field37:29for funders to foot the bill.37:31We need senior and junior colleagues37:33to check the suitability of responses.37:36I think this is asked by Nilmini.37:38Okay, she has left the room.37:40Andra, Andra is here.37:41So, okay, we can't really reliably tell.37:44We have no baseline to measure.37:45Correctness is hard to judge.37:47Okay, Andra, do you want to maybe talk about this more,37:50given whatever you've heard so far,37:51any thoughts, any reflections?37:57Sure, thanks so much.38:02Yeah, I'm still going to take this back38:04to my engineering team38:06and see like how they can use the tool you shared.38:11I was not aware about that tool, by the way.38:13So thanks so much for sharing.38:16I think it's something that we can assess.38:18I love the question that was asked a bit earlier38:23around the difference between LLMs,38:28like the LLMs evolving38:30or having the agent adapt to new versions of the models.38:37For example, in Claude, you have an Opus 4.738:40and then 4.8 and 4.9 coming at some point, I guess.38:44How do you evaluate the code quality, for example,38:49between those kind of releases?38:54And is there anything you do at the moment to capture that?39:00Yeah, yeah.39:03At the end, the code quality, again,39:05is some form of text reply, right?39:08Like, do you think of the dimensions39:11that you want to evaluate?39:13Like, do you have that in mind?39:14What are the dimensions across39:16which you want to evaluate your response?39:21We look into our CI, for example,39:26and like passing that CI pipeline.39:30We have some unit tests that we put in place39:32and other end-to-end testing.39:34So we usually pass the code through those guardrails.39:40But it's more difficult on the kind of edge case scenario39:50or other areas that we might have missed when building it.39:57Yeah.39:58Are these like, for your use?40:00In this case, are the outputs verifiable by any manner, like automated, in an automated way? Are they verifiable?40:08Like, do you run a code snippet and you get the output and the output has to be a particular value or the other that you can check against?40:18Yeah, it's complicated. Often, yes, depending on the functionality.40:23We usually will have a code reviewer looking at the code and just, you know, checking.40:29And by their knowledge, they can flag if something is missing or if there's a test missing.40:35But if we want to implement bigger functionalities and we do not have the bandwidth of a human reviewer, like to check 10,000 lines of code,40:47what is it that we can put in place to ensure that we do not, you know, break things in between, like, iterations of the code?41:00So, yes, we have the CI. Yes, we have the testing. Yes, we have the reviewer, usually.41:09But what else should we put in place to avoid, for example, engineers reviewing nevertheless 10,000 lines of code41:22and be confident that the code matches, you know, the context, the documentation, the testing pipeline and so on?41:31I'm getting a bit like into the weeds of the technical development workflow, but that's where we are struggling at the moment.41:39Like, how how would we manage to do that?41:44And I'm wondering if this kind of tool can help and how, considering the volume of text that that multi-agent system generates your code base.41:58I see. I see. OK, maybe one small suggestion would be and maybe you're already doing it,42:04that if you are using Claude to write the code, maybe use a different agent to review it.42:09Just add some diversity in your reviews. I found it to be helpful.42:14Yeah. And and maybe so that is one. So you talked about one agent which is writing the code.42:20Right. And in your question, you but you also asked the agent that you are building for the end user.42:27Right. Like you're building an AI agent for the end user. Did you have any questions around that?42:34Because you said you don't have any baseline to measure against. There is like the correctness is hard to judge.42:41Any questions or reflections on that? Yeah, definitely.42:48So we haven't built that agent for the end user at the moment.42:52We are thinking about it. Basically, we would like to know if the functionality that's generated matches the needs of the person who express those needs.43:08Yeah. So it's kind of, you know, at the beginning, a user or a program or a health manager writes down a need in plain language.43:17The multi-agent system builds that functionality to the different agents like the research and the coding, the testing, the verification of the code and so on.43:27And that gets shipped. How do we evaluate that? The tool hasn't like broken between the different stages.43:35And this matches what the user has like expressed as a need.43:42Yeah. Yeah. Yeah. I think like what did you think of this idea of like creating the unit test to capture the ideal scenarios first?43:52Because just because you don't have an agent out there right now. So you may want to begin with the ideal scenarios first.43:58Any thoughts or reflections on that?44:04Yeah, I think this goes hand in hand with the question I asked in the forum on how big of an effort it is on a small technical team to do that.44:13And how much knowledge of the like.44:22For me, it's still very unclear who would build those unit tests because the person writing the need, they have all the domain knowledge.44:31But the person writing the code, they have the coding knowledge and maybe not so much the domain knowledge.44:36So is it like a collaboration between the two, these two teams?44:40Like how do you like how do you who is writing the unit test in the different programs you are working with?44:46I think that's my question. And how much you mentioned that this those are not like necessarily for technical folks only.44:53Yes. Yeah. Yeah. That's the intent that we sort of on like maybe the technical folks are just required to make the initial connection and integration.45:02But then you hand it off to the domain experts. Right. Because as you said, the person who's writing the code may not know what is right for the end user.45:10Right. And so it's important to empower the domain expert to do it. Right.45:14And that's the minimal version that we have been discussing that, hey, we don't need to think of all the edge cases or the complex scenarios to begin with.45:24Let's start with the ideal ones and let's maybe focus on empowering the domain expert to do it.45:32Right. And because their job anyways is going to be monitoring these responses.45:38Whatever response quality comes in, the software engineer will not be able to say if it's good or bad is what I'm assuming for your use case as well.45:46So anyways, they would want to monitor the responses. And yeah, you would want a way where the domain experts can feel involved without being overwhelmed or feeling that this is technically out of their reach.45:59Yeah. Does that sound good?46:04Yes, that's exactly it. And thanks so much. Just by looking at it, it does feel technical for some.46:11If I think about stakeholders within the community, this kind of level can feel technical.46:19So I was just thinking of what would the best collaboration be there between the engineers and then the knowledge experts.46:28I don't want to take more of the time, but that's really exciting.46:32No worries at all. Maybe often what at least we see that it's often like a Google sheet where the software engineers just sort of create a nicely viewable Google sheet that is updated on a regular basis.46:46And the domain expert only sees the Google sheet and they have like some dropdowns and fields that they can populate to begin with.46:54Maybe. Yeah. Yeah, that's good. Thanks so much. Awesome.47:01Anyone else? I know we are on time now, so I'm happy to take the next few minutes to answer any questions that you might have.47:16OK, I think if we are all. Pedro, do you want to?47:21No, I do not have a question. I just want to thank you for your presentation and explanation. Thanks a lot for that.47:28Thank you. Thank you so much. I was with my camera off because you were sharing, so it wasn't important for me to show my face.47:34But thank you a lot. I was following everything. Thank you so much. Thank you.47:40Happy to be of help. Awesome. Thanks a lot, everyone. Thank you for being here. Thank you for spending the hour.47:46Hope it was helpful and beneficial in any way. All the best.47:53Bye bye. Thanks again. Thank you so much. Bye bye. Thank you.