AI 101

A visual guide to understanding AI for knowledge workers.

TranscriptCopy transcript

0:00presentation in case you want to use it yourself.0:05And yeah, so we kind of designed this presentation with, you know, a working0:10professional audience in mind, you know, it's not meant to be a hype talk or two in0:15the weeds, but meant to give you a bit of context and guiding principles when0:20thinking about this new tool.0:22So I always like to underline the, how big AI is like in terms of human history.0:30So, you know, please bear with me for a bit of a history here.0:37So in terms of tools, like, you know, tools that human use, probably one of the0:42most impactful has been the human, the hand axe, you know, it was used by the0:48early hominid species and, you know, anthropologists say, like, the use of0:53this tool and tools like it are what spurred our evolution into Homo sapiens.0:59So it's important to think about, you know, because it helped us, it helped every1:03given human do more.1:04And that's kind of a pattern we'll see with tools like this.1:09So that was like, you know, about 2 million years ago, but fast forward to1:141440, and we, I want to highlight the printing press, because before this time,1:21you know, most of the population was illiterate, you know, we couldn't read.1:26But because books were so expensive to create, you know, a tribe would spend1:32weeks, months copying over from a master book, writing, writing new books over and1:40over again.1:41And this just meant that literacy and books were for the elites.1:45But with the printing press, it became much cheaper to produce, you know, books1:49and satisfy the human desire to learn more about the world, you know, what's1:52going on in different parts of the world.1:55So literacy became more pronounced in the population.1:59And with that, you saw major, like, societal movements, you know, including2:03scientific revolution.2:04Then we fast forward to the internet, you know, late 20th century.2:09What the internet did was, you know, a lot, you know, what made the world like2:14much smaller.2:17Just like the printing press, like people are able to produce content and ideas2:20are able to come out there, including things like, you know, e-commerce and2:24social media.2:26So it enabled information sharing at a much greater scale.2:30Now, let's start actually getting into AI.2:33So one of the most important, like, you know, bits of history for AI was when2:37this paper called Attention is All You Need was published by a couple of Google2:41researchers.2:43And, you know, what it may be some of you remember, like using Google Translate2:48before 2017 and then after.2:52But if you do, you'll notice like it got got much better.2:56And it was because of this fundamental technology called the Transformer, which3:00was a type of neural network that paid instead of just translating words, it3:07paid attention to the context words existed.3:09We'll talk more about this.3:11So then five years later, ChazGPT launched.3:16And now we're continuing to talk about history because it became the fastest3:20growing consumer app in history.3:22It became the fastest growing consumer app in history, eclipsing TikTok and3:26Instagram because it reached like 100 million users in just two months.3:30And this is when this technology, this Transformer neural network technology3:34moved from a research lab to the public consciousness, to everybody's phones.3:40And then just last year, another history breaking technology came out called3:46PlotCode.3:47And it involved AI from just answering questions to actually doing work.3:54So, you know, this type of role, software engineers who build software.3:58It was able to do exactly the kind of work software engineers do to the point4:04where, you know, people are questioning whether the job of a software engineer4:08will exist in the future.4:11So, yeah, we especially moved to the doing error last year from just the chatting4:16error with these tools.4:19And, you know, we're talking about numbers and science, but I want to kind of4:24highlight a new story that just kind of demonstrates what people can do with4:31this tool now.4:33So there was this guy, Australian guy, software entrepreneur called Paul4:36Cunningham.4:37And, you know, his dog got cancer.4:40And so he was giving it a certain amount of time to live.4:43And, you know, he wanted to do something about it.4:47So he used a collection of these AI chatbots from Chatbot, Chazipute, Gemini,4:51Grok.4:53And over the process of like, you know, a couple of months, he created his own4:57like mRNA vaccine.4:59You know, he got his dog's genome sequence.5:01He did all the research.5:02He did a lot of learning.5:04And this is all by someone who has no background in medicine.5:07So he learned everything he needed to to create this mRNA vaccine.5:11And, you know, deployed it for his dog.5:14And the dog's cancer started to recede.5:18So, you know, this is exactly the kind of like, you know, use case that people5:22are able to do now, given that they have enough agency.5:26So I want to highlight this quote by Paul.5:29He said, The chatbots empowered me as an individual to act with the power of a5:33research institute.5:34Planning, education, troubleshooting, compliance, and yes, real scientific5:38design, working and converting genomic data into a vaccine prescription.5:43But they worked alongside humans at every step.5:46The combination is what made it possible.5:49So, yeah, you know, I love that story.5:50So I like to highlight it.5:53So now let's get into how this AI, you know, actually works.5:57So you can get rid of AI.5:59So, you know, as you will know, the fundamental like feature of this AI is6:05that it generates text.6:06It generates human-like text.6:09But, you know, to understand the text, it generates it and it has to segment the6:15text in a certain way.6:17So it generates what's called tokens from text.6:21So with this English sentence, it's tokenized in a fairly understandable way.6:27One token for every word.6:29All right.6:31So the way these models are trained, it's trained on a bunch of text, mostly6:37the Internet, you know.6:39They take a lot of, you know, textbooks that exist on the Internet to help the6:45transformer model understand the context that each word is used, you know.6:52Like take, for example, a word like agency, you know, it shows up in many6:57different, in a few different contexts.6:59And we can probably, you know, group it in two different contexts, which I'll7:02get into.7:04All right.7:06So what the transformer is doing is predicting the next token, or to7:11simplify, we can say predicting the next word.7:14So for the example of agency means, you know, when it's predicting the next7:19word, it might predict like government body that, you know, agency means a7:23government body.7:24But, you know, if you change the sentence a little bit to have human in front7:27of it, like human agency means, then it's more likely to predict the next words7:31will be personal autonomy.7:32So in that, it's able to understand a different context that words are used.7:36And that's part of what makes it generate human-like text.7:40But note that this is still a prediction at the end of the day, you know.7:45The way we generate text, we reason a bit more than this.7:48You know, this is just predicting what will come next based on what it's seen7:52in this training data.7:55And yeah, as part of that, you know, these models don't understand like facts.8:01They're just predicting the next token based on patterns in the training data.8:08And so, you know, you might prompt an early version of this model, who founded8:12the Red Cross?8:13And it might give you the right answer, you know, that Henry Durant founded the8:16Red Cross in 1863 because it's cleaner in the training data, not because it's8:20reasoned about it.8:21So that's very important to understand.8:24And here's an example of where that can go right.8:27You know, you asked the question about the 2019 Nairobi Declaration.8:31And there was a Nairobi Declaration, but it wasn't about climate finance.8:35But it confidently outputs text thinking like there was a Nairobi Declaration8:40about climate finance and making up percentages and even dates and things.8:47This is a hallucination.8:48This is what we call a hallucination.8:50The model is just confidently outputting patterns based on what it's seen in the8:54It's not reasoned about it like, hey, is this true or not?8:58So, yes, you know, hallucination is not an unsolved problem.9:04So, yeah, I want to cite the study that was actually just published last year9:08where they asked medical queries of, you know, different open air models with9:16and without source documents.9:17And by source documents, it just means like asking the model to refer to a9:22bank of documents to inform its answer versus just answering directly.9:28So without, you know, this knowledge base, you know, the hallucination rates are9:33very high, you know, unacceptable in any domain.9:37But by simply just asking the model to refer to a set of authoritative documents,9:42they were able to bring the hallucination rate down to zero.9:46And so this pattern of asking the model to refer to a knowledge base or, you know,9:53modernly refer to tools like web search and other things like that has practically9:58eliminated this problem of hallucination.10:00Obviously, we still need to write evals, which I'll get into.10:04But there is a way to mitigate this hallucination problem to make it near zero.10:09Anyhow, you know, an important thing to highlight is that the way these models10:14were built, they were built to support Western languages the best because that's10:19that's what builds up the Internet.10:22And that's what the researchers and the team behind these models speak the best,10:26you know, mainly English.10:29So, you know, the example sentence I gave earlier, you know, we enhance human agency.10:35Here's the same sentence in Swahili.10:38And, you know, it even has the same number of words happen to have the same number of words.10:43Let's see how this gets tokenized compared to English.10:47As you can see, English gets tokenized in a reasonable way, you know, by by word10:52and for complex words, it might get tokenized by suffix and prefix and things like that.10:57But Swahili is tokenized in an incoherent way.11:01You know, if you know Swahili, like this is not these tokens are not logically grouped,11:06you know, it's not by word, it's not by parts of word or anything like that.11:10And, you know, like I said, this is because the models are used to dealing with English11:14the best. But this has important implications for performance and pricing.11:22So, yeah, yeah, here just for that same sentence translated into different languages,11:28we're counting how many tokens get produced.11:31So, you know, English is four, but and French and Spanish, not that much more.11:37But as we get into non-Western languages, you know, throughout Asia and Africa,11:43you notice that they have more tokens that are produced for the equivalent phase.11:48And because these models are priced by tokens, they charge you based on how many tokens11:55in the input and how many tokens in the output.11:58It means that, you know, to support these AI, to use these AI models for non-English languages12:03is going to cost you a lot more, basically, you know, especially by the multipliers you see here.12:08So, yeah, you know, just a simple example, like if you're building a maternity ward health chatbot,12:13as we see in the among our grantees who are piloting AI.12:17For a given set of queries, thousand queries, you know, it might cost like 42 cents to do them in English,12:24but it will cost over double the price to generate those responses in Swahili.12:32And, you know, it's not, you don't get the same quality with this12:35because these models have an important phenomenon called a context rut.12:40So, they have, let's say, like a limited amount of space, like working memory.12:44You can think of it as working memory.12:46And the more that is filled up, the worse the performance is.12:52So, if you're supporting a non-English language and, you know, just because it's not in English,12:57it just means more of that working memory is taken up.13:00That also means the quality will be worse.13:04So, that's important to think about.13:06Yeah.13:07What was the phrase you just used to describe that phenomenon?13:10Context rut.13:11R-O-T.13:13That's super helpful.13:14I've been noticing that in flawed code that, like, after 150,000, I've been calling it dumb mode, but this is...13:22Yeah, yeah.13:24And then, you know, just as a quick aside, you know, this is very important, even for English queries,13:29because maybe you notice, like, you've heard the news story last year of, you know, the child that committed suicide13:37because he was talking to a child LGBT, and at some point, child LGBT started, like, encouraging him to commit suicide.13:45It was because the conversation ran on for months.13:49It was one conversation, and it ran on for months.13:52At a certain point, the safety guardrails that OpenAI has to, like, prevent this kind of thing eroded completely.13:59So, performance, like, you know, more context is taken up.14:03Performance can degrade pretty rapidly.14:06But, you know, there's some solutions for that now, especially.14:10Hey, sorry.14:11I have a question, too, if you don't mind.14:13You were talking through kind of...14:15The preface to this was about kind of how the early models of LLMs, early as in, like, two years ago, were being kind of constructed.14:23I'm curious whether, and maybe you're going to get to this, but whether what you're talking about now is sort of rooted in the early model design14:30of kind of the tokenization in different languages and everything,14:33or whether, like, that's also as relevant today with some of the more advanced models.14:39Yeah, it's relevant today.14:40So, with the more advanced models and, you know, ways of working, they're kind of built on top of this early model.14:47So, we'll get into that.14:51Okay.14:52So, as I've described these models, they're predicting the next token.14:56They're just guessing what the text comes next.14:59But there's some work that these model labs do to make them like an assistant, you know, to make them more safe and easy and friendly to use.15:09So, I want to quickly touch on these.15:12So, yeah, you know, like a freshly portrayed model is a prediction engine.15:19And so, the answers to a lot of, like, realistic queries will not be as useful, you know.15:26If somebody asks, like, how might you treat a burn?15:28Just a pre-trained model might just say, hey, burns are a common injury that can result from heat.15:34The word burn, you know, will get us to the etymology.15:36It will not be answering the person's question because you can see how this is, like, based on these patterns of where it's seeing the word burn.15:45It's just kind of predicting, like, what the definition is and not being actually useful.15:50So, there's a technique called instruction tuning that simply you just give the model a bunch of question and answer pairs to get it used to answering questions.16:02Because as it was trained before, it was used to just predicting what the next word would be.16:09But that's not useful, you know.16:10So, to make it useful to answering humans' questions, you just feed it a bunch of QA pairs.16:15And this is called instruction tuning.16:18But, you know, this version of the model, the instruction tuning model, will answer any question.16:25If you ask it to make a bomb, it will answer, you know, it will tell you.16:28And so, it's seen as unsafe, you know.16:30So, there's a bit more work to do to make it useful and safe.16:35So, the final layer of, you know, post-training that's done with these models is something called reinforcement learning with human feedback.16:42And what it means is that you're giving the model feedback.16:46You're getting the model to generate, like, a few different responses.16:50And then you're reading the model's responses to tell it, like, hey, this was better than this.16:55And so, it is, in a diffusive way, learning what a good answer should look like.17:02And, you know, the research model uses this technique to prevent it from giving, you know, medical advice or financial advice or, you know, answering unsafe queries, like how to make a bomb or, you know, how to commit suicide or something like that.17:16And, you know, this makes it generate the responses that you might be used to today.17:22You know, responses that maybe go over a few options but will not go too far, you know, will not answer all queries.17:30It seems to use some kind of judgment.17:32So, that's important to touch on.17:34But, you know, the safety training is actually mostly done in English, you know.17:41So, you know, if there's an unsafe query, like, you know, how to make a bomb, it might refuse in English.17:48But if you ask it in a lower resource language, you know, Amharic, Swahili, it might actually answer.17:55Because the reinforcement learning with the human feedback stuff is kind of an expensive process that relies on data annotators and people with expertise on certain topics.18:03So, it's mostly done in English.18:05So, this is an important thing to caveat.18:07And also, you know, the people that are doing this reinforcement learning with human feedback are primarily, you know, from the global south.18:15There was an industry of using data annotators by, like, companies like Facebook for, you know, labeling offensive content and things like that.18:25And those same industries have been reused to train these models.18:28And, yeah, you know, there's an important, like, you know, labor dynamic here.18:36Okay.18:38So, I want to get a bit into, like, one, there's how you can use AI in your daily work.18:46But, you know, as funders, we're getting these proposals where people are proposing, you know, different AI interventions and how they're going to use it in their organization, how they're going to use it to impact their beneficiaries.18:57So, I want to talk a bit about, like, you know, in addition to deploying AI, like, how do we evaluate it?19:05How do we think about evaluating AI?19:09So, we launched an accelerator last year, you know, the AI for Global Development Accelerator with OpenAI and Center for Global Development.19:18And we had eight organizations pilot AI interventions, and we were able to work with them, you know, as part of deployment and time and see where exactly to struggle.19:26And evaluation, like, understanding, like, what the AI product they built, if it's actually working, was the top item that folks struggle with, you know.19:35So, we developed this four-level framework to think about evaluating AI.19:40And we'll mostly just get into L1, but I wanted to touch on all of them.19:44So, L1 is, like, before deploying the users, does the AI behave the way you want?19:49Can you compute some metrics of how it's behaving?19:52And then L2 is where we measure traditional product metrics, like, are people using it?19:58You know, monthly active users, daily active users, things like that.20:02And then beyond them using it, L3, we look at, is this positively influencing user behavior?20:09You know, are they learning more?20:10Are they, you know, uptaking more health-seeking behaviors?20:14Things like that.20:15And, you know, the fourth layer, we're looking at if this improves development outcomes, you know.20:21And part of this is because, you know, in RCT, the currency for, like, impact takes a long time and is very expensive to do.20:29So, with this faith evaluation framework, can we think about building successive bits of confidence before, you know, taking on the big bad RCT?20:38And, yeah, you know, just to get into what it actually looks like to do model evaluation,20:43you're essentially defining specific dimensions you want to evaluate your chatbot on.20:48You know, it could be factual accuracy because we know, like, you know, hallucination is a big problem.20:55Or it can be, you know, checking for safety, making sure it doesn't, you know,20:59recommend specific drugs or prescriptions or things like that to use where a medical professional should be used.21:07And given that a lot of this level of model evaluation is related to safety.21:16Sorry, let me drink.21:20Yeah, this level, most of this level of model evaluation is related to safety because of the liability, you know.21:30Like, if somebody takes the wrong medication because of, you know, what the AI chatbot recommended21:38or goes and buys certain stuff because of what the AI chatbot recommended,21:42generally, you want to avoid, like, having the AI chatbot do that just because of the legal risk.21:49And, yeah, here are some examples of, you know, these kind of, like, level two and level three metrics.21:55If you have some kind of onboarding floor, specific tasks users are supposed to do with the help of the chatbot,21:59you should measure that. You should treat the chat experience like a funnel and measure where users drop off and things like that.22:11Okay, so let's skip these slides, but just get into that.22:15Hey, we deal with evaluation thoroughly in this eval playbook that we created with the Center for Global Development and IDinsight.22:26And another thing that came up a lot with a lot of the awardees building with AI is, like, how to deal with data.22:34Given that some countries are, you know, want data to be processed in-country,22:40it's important to think about, like, when users are talking to these AI systems, what's happening with the data?22:46And it was the case that up to 2024, yeah, the servers that are processing these AI responses are mostly in the U.S. or in Europe.23:01So for Africa or India and organizations building with these who had requirements to be in-country,23:06their only option was, you know, using open models, open source models that they could host themselves.23:13Jacaranda Health, one of our grantees, they went down the path of fine-tuning LAMA23:18because, yeah, the off-the-shelf models that you have servers in Kenya that they could use are even on the African continent.23:26That's changing, you know, now there are providers like OpenAI have in-country affordances in India.23:35But, yeah, you know, it's yet to be the case in Africa yet.23:39And given that the beneficiaries, when they're talking to these chatbots, and it's a personalized experience,23:44they're likely to reveal a lot of, like, PII, personable identifiable information, very sensitive information,23:52from things like their address, where they live, to, you know, their medical status, you know, their HIV status,23:59their bankruptcy status, things like that.24:02As we've seen, like, a lot of people confide in these tools, you know, because they see them as human-like, but not human.24:10Yeah, yeah, go ahead.24:13Sorry, I didn't mean to cut you off mid-thought, but this is just so useful24:17because I think we've had maybe two conversations just this week on this topic.24:22And it's useful for us both from thinking about what questions to ask our applicants and potential grantees,24:27but also in the way that we're deploying tools internally.24:31So to just give you an example, Michelle on the call and Alex, who unfortunately just had to drop,24:36are thinking about ways to pre-draft email responses to our inbox.24:40Our inbox used to get completely flooded before, and we would just get really behind on replying to it.24:47And, you know, it's not obvious how we should treat that data, right?24:52Because emails often have signatures that include home addresses, like you mentioned,24:57certainly email, phone number, and name, and title, and stuff like that.25:02So I'm curious if you have any, like, resources you could point us to or, like, established guidance documents25:08that can help us, you know, think about that.25:12Yeah, so there are these tools called, like, PII Scrubbers.25:17And, yeah, they actually use LLMs under the hood to detect cases when, you know,25:24any kind of, like, message or inbound message has, you know, PII and then, like, automatically scrub it.25:29And, you know, that's one option, you know, and there are some good, like, open source solutions for that.25:37But a more common option is to have a query rewriting step, you know.25:44So if you're using AI, like, to kind of process inbound like that, you can just have a step where, you know,25:50you ask it to strip out the PII.25:54There are also these, like, tools you can use.25:57Anthropic uses a tool called Climb, and it does a lot of this for you.26:02It's a paid tool, but it does a lot of this for you, such that when you're,26:06you want to look at user data in a way that's decoupled from the PII,26:10maybe that's not super useful, right?26:12Maybe some light demographic information would be useful,26:14but you don't need, like, all the PII that people usually put back in proposals.26:19And they do all this for you.26:21The tool that Anthropic uses is called Climb, but I've mentioned there are other competitors to that tool.26:27There's also some open source free tools you could use if you have a little bit of capacity to, like, you know,26:33deploy your own models and software.26:36Yeah, and this is a bit of, like, the kind of guidance we give to awardees, grantees who are thinking about, you know,26:42choosing between open models, open source models, or closed models, you know, like OpenAI Anthropic.26:49And, yeah, you know, it's just kind of like a checklist, including how long the data will be retained,26:55how long the data will be stored, and then, you know,26:59And, yeah, you know, it's just kind of like a checklist, including how long the data will be retained.27:05OpenAI, by default, retains data for 30 days for abuse and safety monitoring purposes to check, like,27:11who's asking about writing a bomb, creating a bomb, or something like that.27:15But they do have something called zero data retention, and the other model providers have it too,27:21where, you know, if your use case is sensitive enough, health domain especially,27:25you can tell them, like, hey, don't ever retain my data, even for this abuse and monitoring purposes.27:30So, yeah, you know, even if walking through this, your requirements are pretty strict,27:35there are ways to work with these model providers so that they can satisfy it.27:40And, yeah, you know, I just talked about some of these, you know, using these PI scrubbers to redact the models using open models.27:53But, you know, one of the most important things for this is, especially during the pilot,27:59you can just have a – where people don't know it's an AI quite yet, you know,28:04obviously you want to disclose at some point, but you can just, like, have the AI draft the response,28:09but an actual human, you know, the helpdesk worker or whatever, whatever front-end worker,28:14actually be the one that sends the final response.28:16So, there's a step where they have to use their judgment, and you can implement an assent operating procedure28:22where they don't include the PI or anything like that.28:26Okay, so this is kind of a bit about the implementer perspective in our world, you know,28:30when they're building and how they evaluate AI.28:33But now I want to get into more, like, you know, how us as knowledge workers,28:38like how we can best use AI and, yeah, you know, the different types of tools available there.28:44So, you know, there's different levels of, like, talking to this AI chatbot, you know.28:50There's a phrase, you know, bad data in, bad data out.28:57And the similar applies to AI in that if you write a vague, poorly defined prompt,29:04you know, the answer is not – is going to be vague and, you know, less defined.29:10And, you know, as we see in this example.29:13But if you put in a bit of effort to create a detailed prompt and treat the kind of AI,29:22like it's sort of an intern, you know, that's pretty capable,29:26but maybe doesn't know your domain-specific language and, you know,29:30the kind of things that you look for in a good response.29:32If you can define what good looks like to an AI, you will get a better response.29:37So here's a general structure, like the industry has come up with, you know,29:41that, you know, you should tell the AI to kind of role play a bit.29:46You know, it can be a specific role or it can just be like, hey,29:49you are the best software engineer in the world.29:52You know, that actually does help a lot in generating better responses.29:59And another thing is, like, you know, being pretty detailed about what you want to see30:03in the response, you know, asking for specific things so that the AI, you know,30:08go off and try to fill it in.30:10If you kind of think about it like as a prediction machine,30:14then think about this as like giving the AI a form, like, you know,30:19treating your prompt like a form and then the AI just fills it out, you know.30:23That pattern does help a lot and leads to much better responses.30:26And yes, so something we've talked about before is just that30:31asking the AI to refer to a bank of knowledge helps a lot in mitigating30:37hallucination and also generating better responses.30:43And part of this is because when the answer is not in the knowledge base,30:49it will refuse to answer.30:51You know, because we structure the prompt like this.30:53So this does mean you would have to invest in or you would benefit more30:58if you already have this well organized and structured knowledge base, you know,31:04of the information in your organization.31:12And something I wanted to highlight here is that, you know,31:15sort of like doing math homework back in the day, you get extra credit.31:20You get better responses from asking the model to take step by step.31:25So before a class of models called the reasoning models,31:30you would have to ask, you would have to ask this explicitly.31:32You have to tell the model explicitly, like, hey,31:34think through this step by step.31:36But now with these reasoning models,31:38you have to ask the model explicitly.31:41All right.31:42So now I want to get into like agents and agent workflow.31:45So, so far, you know,31:46what we've been talking about is essentially a chat experience.31:50But maybe we add specific affordances like, you know,31:53a knowledge base and things like that.31:55But I want to look like I want to look at a new class of AI.31:59I want to look at a new class of AI.32:01I want to look at a new class of AI.32:03I want to look at a new class of AI.32:05I want to look at a new class of AI.32:07I want to look at a new class of AI where you give the models agency32:13or like autonomy to, you know,32:16use different tools and then come up with a response.32:22Yeah.32:23So, yeah, you know, the reasoning models,32:27you just gave it a prompt and you got a response like a simple chat,32:30chat interface.32:32And there's a step a bit more involved in that.32:36You can create something called like an LLM chain where you,32:40you actually have multiple prompts to do certain things.32:43Like you could have a prompt to do research,32:45then a prompt to draft the response,32:47then another prompt to review the response to make sure it's sure it's32:50okay.32:51And this is getting a bit more human.32:53Like, you know,32:54we're going from prediction to being more more system like,32:57and this is one of the ways that,32:59that happens.33:00So we call this an LLM chain when you chain together multiple prompts to,33:03to come up with an answer.33:06Then there's something called an LLM workflow that you can,33:11you're not quite given the model autonomy,33:13but you're introducing things like branching, like, Hey, you know,33:18if the result, the result of the first prompt is this or that, you know,33:22use this prompt versus that prompt.33:24And then you can have things like a human gate.33:26So when it, when your LLM team becomes a bit more complicated like this,33:30when you're using branching and human gates,33:33that's what we call a workflow.33:37Then now we have like an agent.33:40And this is very, very important to design.33:43An agent is just an AI system that has agency that thinks in a loop,33:49you know, it, it plans what to do.33:52It plans what to do.33:53It takes actions or observes the result of these actions and decides on the33:57next step.33:58So this loop is the one part of like what makes an agent an agent.34:03Another part is the ability to use different tools, you know,34:06that it can search the internet.34:08It can write and run code or it can run SQL in,34:11in a database and things like that.34:13And this is definitely getting a bit more like human-like given that it has,34:18it has agency.34:19And maybe that's a bit of what, what makes us human.34:24Now with agents,34:26it is much more expensive to have an agentic system because where previously34:31you're just talking about question answering, you know, one to end prompts.34:37Now you're talking about operating in a loop, you know,34:40and depending on the budget,34:43what we call the thinking budget you give to the LLM, it can run all day.34:47You can run for a week in the loop and it's incurring costs all the while.34:52So, you know in January,34:54OpenCloud came out and people were, you know,34:57deploying this as part of their part of their organization and create like an35:01AI employee.35:02And it was incurring like thousands of dollars every day, you know,35:06to just have, you know, a couple of AI employees.35:09So when you produce agents,35:13it does incur costs and it incurs more complexity.35:20And yeah, you know,35:21part of what enables agents is what I referred to before this kind of like35:26chain of thought, you know,35:28asking the agent to think that step-by-step and this is what we call reasoning35:32mode. If you've ever like read the book,35:35Thinking Fast and Slow, you know, he talks about, you know,35:40this kind of like more primitive style of thinking where it's like fast35:43instinctual, you know, like jumping right before like a car, you know,35:47drives past you and things like that. And then deep, slow thinking,35:51like analytical slow thinking.35:53So this was kind of baked into these models where models have a fast35:58mode and they have a reasoning mode and the reasoning mode is what enables36:02these kinds of agentic use cases. You know,36:05the reasoning mode has another name that's used in the news quite a bit36:08called a test time compute.36:12So there's different architectures, right? From one shot,36:15the LLM chain to a LLM workflow to, you know,36:19this full agentic system. And, you know,36:22you have to be scrupulous about like what to use where, you know,36:26for simple low stakes use cases, you could just use, you know, one shot,36:31you know, just asking a question and getting the answer.36:34For more complicated use cases that are low stakes,36:36you might use an LLM chain,36:38but for things that are high stakes where you want a human involved or you36:41want to do different things, depending on the output,36:43you want an LLM workflow and then or complicated and high stakes use case36:47where you want human input, but you also want the agent to try his best,36:51you know, to exhaust all its resources as possible.36:54Then it's when you want an agentic system.36:59And yeah, you know, I wanted to reason about this,37:02like it's a form of a pyramid, you know,37:05that most people are starting off using AI through these chat interfaces,37:08Cloud AI, Chat2BT, Grok, and they're doing it in a very one shot way.37:14But as you start to use this more and more and you're building, you know,37:17workflows, you might find yourself like, Hey, you know,37:21asking the same thing of the agent over and over again. And instead of,37:24you know, just typing over and over again, you might have these like prompts,37:27you know, these skills, I mean, repeatable workflows.37:30And we'll get into this.37:32And there are these autonomous agent harnesses now like Cloud Co-Work,37:36Cloud Code, Codex. And then at the top, you know,37:40are these always on agents that are running all the time.37:44And then you can chat over WhatsApp or Telegram too, to actually do work.37:48You know, they have their own computer, they have, you know,37:51their own tools that they can use.37:53So that's kind of like the peak of what we call like the agentic ladder right37:58now.38:01Okay. So, you know, Cloud Code and Co-Work,38:06they popularize this idea of skills that there are things that, you know,38:10you might do over and over again, like, you know,38:13a software engineer might do a front end design over and over again,38:16or a database design and things like that. So, you know,38:20there are prompts that you can save somewhere, you know,38:24is this a prompt that's saved somewhere that when you need it,38:27it's easy to invoke, you know?38:32And, you know, the prompt is not just text.38:35You can give it the ability to run certain tools. You know,38:39if you're building a system and you write a skill,38:42you can give it the ability to search the internet, you know,38:45or you can restrict its ability. Like, okay, this prompt is,38:50this skill is just for reviewing applications.38:54It doesn't need to,38:56it doesn't need to save any new file or, you know,39:00research the internet. It should just be about reading the proposal.39:06And the reason why this is useful is,39:08is because the more you're using these tools,39:13the more you're identifying these repeatable workflows.39:17It's just the point where, you know, you, when you write a skill,39:20you might use it, you know, a few times,39:23but then you might realize the response is not what you want.39:26So you're constantly improving it to make it better for next time.39:30Very similar to like, you know,39:33how you would treat an employee that, you know,39:35maybe when they get started at the job, you know, they don't know a lot,39:39but as they start to generate like output and do work, you know,39:44you might give them a little correction and they will become better over time.39:46So the idea, we call this like, you know, compound engineering that,39:50the more you invest in the tools and skills of these agencies,39:55the more you will benefit, you know.40:00And yeah, you know, at the top of the pyramid, we talked about, you know,40:04the systems where you give it a lot of agency, you know,40:06these like open call agents that have their own computer, they can, you know,40:10post in Slack, they can read your Google workspace.40:12You have to, you have to work your way there.40:15You should not just jump there on day one. You know,40:18you have to work your way there through the regular use of these tools that in40:21front of the repeatable workflows and the things that should have access to,40:24because it can go, there are many situations where it can go around.40:27The biggest right now is, you know,40:30this Meta-AI security researcher who's open call agent started deleting a40:35bunch of her emails, you know, it just went broke.40:39And right now in security, it used to be that PHP apps,40:45you know, there's a certain kind of web app called a PHP app,40:48where that's what these hackers were focused on the most.40:53That changed rapidly, like in the past month.40:57Now what these hackers are focused on the most are these open call systems,41:01because as you can imagine,41:03if you have these open call agents running on your computer and they get41:07hijacked somehow, you know,41:09they do this technique called product hijacking,41:11it can lift a lot of organizational secrets or personal secrets to be hackers41:16in a way that, you know, it was just kind of like not possible before.41:20So that's why I say you have to work your way to these like fully autonomous41:24systems and you might not even get there, you know,41:27because you want to retain some human agency at the end of the day.41:31And yes, so with that, sort of like what Paul said, the quote from Paul,41:36you know, the best pattern is not full autonomy.41:39You know, you should have this compound engineering pattern where,41:43you know, as the agent generates outputs,41:47the human reviews it and bakes it into the skill so that the next run is41:51better versus, you know, handing things off to the agent.41:54And also just expecting the agent to get it right the first time.41:57Okay. So, so I want to close with some guiding principles.42:02You know, one is that, you know, in working with AI, like, you know,42:06always treat the output as kind of a first draft, you know,42:10I think there's the mode of thinking sometimes where like, you know,42:14it should just do it right the first time, you know,42:16and maybe that's what these AI companies are kind of like marketing,42:20you know, that it's just going to do it right for you the first time.42:23I'm not sure that's ever going to be the case.42:24You know, it's a multiplier, you know,42:28it's going to help you work your way to like the final output much faster.42:33But I don't think we should expect a place where it generates it42:37correctly the first time.42:40And instead of a prompt, instead of a text, you know,42:44a line item, like you should think of it as a task.42:47What tasks are you giving to the agent?42:49And that forces you to kind of like think about, you know,42:51a lot of the checklists that I've outlined here, you know,42:54what role you want the agent to play,42:56what context you want to give it to,42:58what format you expect the response in,43:01the constraints, and maybe you give it a few examples,43:04you know, of related problems.43:06And this is very helpful, right?43:09Because this mode of thinking,43:12you don't lose your own agency by ever relying on these tools too much.43:17You know, there's a part of it,43:18like an education report that just showed like, you know, in college,43:22in college now.43:24Yeah, I imagine a lot of us are like, oh, man,43:27I wish I had this tool in college.43:29But actually the effect right now in college is that it's making people,43:32you know, get worse grades, right?43:36Because they're relying on the AI too much.43:38And then when it comes to the interview or the tests, you know,43:41they're not able to perform as well.43:43So this idea of both collaborating with the AI43:46and, you know, getting it to do tasks that you synthesize43:51and do the ultimate generation,43:53I think is very important for maintaining our own agency.43:56And yeah, you know, depending on the stakes of the domain,44:00you want to match that one level of AI autonomy you're bringing.44:05You know, for simple things, brainstorming ideas,44:09drafting emails, you know, you can almost like,44:12almost give AI a lot of autonomy in doing that.44:14You know, because the stakes are low.44:16But also for very sensitive things like health recommendations,44:20legal or financial advice,44:22it's very important to have a human driving44:25or a human in the loop.44:27And yes, you know, in building AI products that others will use,44:32evaluation is very important to think about, you know,44:35and a phased evaluation pyramid is one example.44:38And yeah, just to double click,44:40having humans in the loop is very, very important.44:44At this stage of AI,44:46the stage of AI is still technically in a probationary period.44:50So it's not AGI,44:53artificial general intelligence that just does everything right.44:56I'm actually not sure if that's something we're ever going to get to.44:59So as of now, we should always think about, you know,45:02existing in collaboration with AI,45:04using it to enhance us versus relying on it too much.45:07Yeah, so I want to close there.45:10You know, at the agency fund,45:11we're starting to think about how our grantees can use AI tools45:15to improve their own cost effectiveness,45:17especially in a time of post-USAID and things like that.45:20So stay tuned.45:22We're going to start a virtual meetup45:24where we talk a lot more about these things45:26and people share real workflows.45:28So yeah, with that,45:30I'm going to pause and take any questions or discussions.45:33Thanks.45:41So, so helpful.45:43Thank you so much.45:45Any questions from the team?45:47I have a few, but I don't want to take the mic first.45:50This was enormously helpful.45:53This is by far the most useful sort of overview45:57I've ever heard of AI45:59and how we got to where we are46:01and what it can do and what it can't do.46:03I'm curious to hear.46:05So I just want to make sure I understand.46:07What I heard earlier was that46:11it remembers what I'm asking it to do46:15for some period of time,46:17and then it resets.46:19Is that right?46:21Go ahead.46:23Yeah, yeah, yeah.46:25That's right.46:27It didn't used to be the case.46:29That's how the suicide thing happened46:31because OpenAI was not resetting.46:33What we specifically call it is compaction.46:36That when the conversation gets to a certain point,46:39it creates a summary46:41and then the conversation starts with that summary.46:43So that's what happens now.46:45That didn't used to happen before,46:47but that's what happens now.46:49Okay, that's helpful.46:51And so then with that sort of new feature,46:53how can we make sure46:55that what we are building on46:57over weeks or months47:00continues instead of starting over?47:02I want to make sure this feels like something47:04that there should be an obvious answer to47:06that I'm missing,47:08but I want to make sure I understand it.47:09Yeah, yeah.47:11That's a great question.47:13So the answer in short is tool calling.47:16So, you know,47:18like OpenAI, right?47:20If you've used ShellGBT,47:22you'll notice it has this memory feature47:24that as it learns a bit about you,47:26it has a separate doc.47:28It's called a memory.md doc47:30that it just writes bits of information about you47:33and it loads that memory47:35into every new conversation47:37and the other providers47:39that are building your own system,47:41you would need to implement47:43your own version of that memory.47:45And as you can imagine,47:47if it's an enterprise use case,47:49you could have a separate memory47:51for each organization you deal with,47:53each employee.47:55So the answer is that47:57you implement a memory system,47:59a tool, a memory tool48:01that it just writes bits of information to48:03so it can remember it for the next time48:05because you're right.48:07Otherwise, if you didn't have that,48:09you'd have to do it every day48:11like what happened, you know?48:13Yeah. Okay, that's really helpful.48:15Super helpful.48:17And then I have a tiny question,48:19but I'm still hoping that48:21you can avoid the enormous amount48:23of rework I've been doing.48:25I feel like sometimes48:27it ignores my instructions.48:29So I'll say like,48:31don't change a single piece of text48:33in what I'm doing.48:35Just do X, Y, Z with it.48:37And then it ignores me entirely48:39because I'm prompting it48:41to more explicitly follow instructions.48:43Like that's one small example,48:45but I feel like that has48:47much bigger implications.48:49Yeah, this was a problem48:51in coding,48:53especially in the beginning.48:55You know, there was this drift.48:57And it could be many different things.49:00One thing to think about49:02if you're using it in a chat use case49:04is like where you're giving the instruction.49:06It's better to give the instruction49:07of your prompt and then just add49:09all the supporting information49:11later sometimes.49:13Or actually like49:15instead of asking right away,49:17just like describe your problem49:19and ask,49:21do you understand everything49:23before I ask my question?49:25Like having a first pass49:27where you ask it to confirm49:29its understanding of the problem space49:31before you give it a task49:33or something.49:35And another49:37thing that may be an option49:39is giving it the ability49:41to verify its work,49:43you know,49:44because especially like,49:46you know, these AM49:48have different personalities.49:50And I think Claude is like smart,49:52but kind of like sometimes49:54a little lazy, you know.49:56So, yeah, it's kind of weird to say,49:58but like it's true.50:00You know, these concepts50:02and personalities and codex50:04is like, you know, a little bit50:05like you,50:06if you get,50:07if there's an opportunity50:08for it to verify50:10if it did the work,50:11then it's more confident50:12like going down that path50:13of doing the work.50:14And in coding,50:15we have these like tests,50:16you know,50:17it can write a test50:18and then run the test50:19to see if it passed.50:20But depending on the domain,50:21you might not have50:22something like that,50:23but that's just something50:24to think about,50:25like, you know,50:26having a verification step50:27that comes later.50:28Yeah, that is super useful.50:30Thank you.50:31Like these,50:32like both small tips50:33and like big pieces50:34of information50:35are both really,50:36really useful50:37in making sure we're using this50:38to the best of our capabilities.50:39Rogers, go ahead.50:43Oh, yeah.50:44And actually,50:45I have like things50:46that I really want to fit in,50:47cut me off and jump in.50:49I wanted to ask a little bit more50:50about context rot50:52because I've definitely50:53seen it a lot in my work.50:54And the way I sort of manage it50:56with the agents now50:57is like sub-agent orchestration,50:59which, you know,51:00is just this kind of fancy word51:02that people are using51:03to describe spawning new sessions51:05underneath the session51:06so that you're using51:07like the most valuable part51:08of the context51:09for each discrete task.51:12It feels a little crude51:13as a solution.51:14I wonder if that's how51:15you guys are using it51:16or where you think51:17that's going51:18and if you think this problem51:19will just be kind of like51:20solved in a year51:21type of thing.51:23Yeah.51:24So,51:25one,51:26on the horizon,51:27the context window,51:28you know,51:29the technical term51:30for the working memory,51:31the size of the working memory51:32is getting bigger.51:34With Claude,51:35it was like 250K tokens.51:36Now,51:37it's a million tokens.51:39But,51:40you know,51:42like the way,51:43like what you just said51:44is one solution, right?51:45Like using sub-agents.51:46So basically,51:47like they fork the conversation51:49and bring back51:51just the answer51:52so it doesn't happen51:53in the main thread.51:54It's sort of like outsourcing,51:55you know,51:56you like outsource51:57a little bit of the work51:58to a separate agent51:59that comes back with the answer52:00so you don't have to think52:01too much about the details.52:02It's kind of like52:03what is happening.52:04But,52:05the most effective way52:06is that you're getting52:07a little bit of a,52:09you're getting used to,52:10like,52:11what amount of work52:12will fit in the context window.52:15Thinking ahead,52:16like planning your work52:17such that it's,52:18you break it up52:19into small chunks.52:20It's kind of like weird.52:21Like how much of this advice52:22is like,52:23you know,52:24like about real life52:25that can apply to real life.52:26So,52:27like,52:28if you have a long52:29streaming task,52:30you have to be able52:31to do it in one session.52:33Break up the work52:34such that,52:35you know,52:36it can fit in smaller sessions.52:38And,52:39you should be outputting52:40a lot of artifacts,52:41you know,52:42like documentation52:43and things52:44so that the next session52:45can pick that up.52:46So,52:47encoding,52:48this is like,52:49you know,52:50with pull requests52:51and all the files52:52that you're editing.52:53So,52:54yes,52:55the long term solution52:56is that these context windows52:57are getting better.52:58With cloud,52:59you don't have to think about it.53:00But,53:01yeah,53:02the solution right now53:03is to like,53:04you know,53:05just do a bit of planning ahead53:06so that,53:07you know,53:08you chunk your work53:09into what fits in a session.53:11Awesome.53:12So helpful.53:13I talked to someone53:14the other day53:15who said that,53:16that using cloud53:17is making them a better manager53:18because they're realizing53:19how they didn't scope53:20their tasks53:21as they would53:22get to their reports53:23perfectly well.53:25Thank you guys again so much.53:26Thank you,53:27Edmund,53:28for all the thought53:29and care53:30that you put into this.53:31Like,53:32as Amy said,53:33like,53:34I do think this is53:35the best of these types of sessions53:36that I've seen by far.53:37And it's really helpful for us.53:39Thanks, guys.53:40And yeah,53:41you know,53:42as you guys start to pilot53:43an AI organization,53:44let's continue to trade notes.53:45You know,53:46I think we're also thinking about53:47how we're playing AI53:48and like the best way53:49to bring everybody on board is.53:50So,53:51we'd love to keep,53:52you know,53:53trading notes.53:54We'd love that.53:55Absolutely.53:56Thank you so much.53:57Appreciate it.53:58This is great.53:59Thanks, guys.54:00Thanks.