# Making state management intelligent

Canonical: https://davidk.dev/talks/making-state-management-intelligent/
Video: https://www.youtube.com/watch?v=kUzHkj5DtPo
Speaker: David Khourshid (@DavidKPiano), creator of XState
Type: Conference talk · Event: Craft Conference 2024 · Year: 2024 · Duration: 42:26 · Views: 157 (as of 2026-10-19)
Topics: AI agents & LLMs, Adaptive & intelligent UIs, State management, State machines

## Summary
Managing state is complicated. Humans are even more complicated. As developers, it's our job to deliver seamless and intuitive user experiences, but the sheer complexity of human behavior and the real world can make this a daunting task. In this talk, we'll explore a radical new approach to app development where language models (LLMs) and reinforcement learning (RL) can be used to handle app logic

## Key takeaways (quoted from the transcript)
- In fact virtually every language has a state machine library and with this you could actually define state machines in a declarative way. (3:00)
- If we graph all of this out, all of the possible things the dog could do, then essentially we're looking for the best path from the initial state to the goal. (26:00)
- One of the main reasons that I'm using state machines for this is because it is very important to realize that in each state of the espresso machine, you could only do a specific number of actions. (30:00)
- Overall state machines and reinforcement learning make LLM agents more intelligent and predictable. (35:00)
- Just like we saw with using state machines with LLMs, the state machine guides the agent instead of trying to have the agents do everything which is unpredictable. (39:00)

## Chapters
- 0:00 Introduction
- 8:00 I identify three main problems with LLMs
- 11:00 The agent could do two things
- 17:00 Going back to Pac-Man, you could think of the environment as…
- 24:00 We have espresso, steam, milk, basically a cortado
- 34:00 We have to guide LLMs using things like state machines to…
- 37:00 Another idea is adaptive user interfaces with reinforcement learning
- 40:00 Try traversing an if else chain or a switch statement and…

## Transcript
[0:00] So, let's talk about David. David is a software engineer who loves playing the piano and is passionate about animations, state machines, cutting edge user interfaces, and open source. Previously at Microsoft, he is now the founder of Stately AI.AI, a startup focused on making even the most complex application logic, visual, and accessible for developers and non-developers alike. So, let's welcome to the stage David Corsid. Hello. Thank you so much for having me.

[0:00] I am so excited that all of you are here because I am going to be talking about something that I am truly excited about and I hope that you are too, especially in the times that we live in where technology is changing. We have AI starting to come into the picture. So, we're going to be talking about making state management intelligent. So, again, my name is David. I'm @DavidKPiano pretty much everywhere online and I work at a start well I'm the founder but I also work there a startup called stately.ai and mostly we focus on the state part and specifically we focus on state machines.

[1:00] If you know me online, you know that's something I love to talk about. And the AI part actually sort of came as a you know, not a happy accident, but just basically I always thought of state machines as one of the most basic forms of AI. So even before LLMs and neural networks and things like that, if you wanted to make, for example, an AI for enemy behavior in a video game, chances are you would use a state machine or something that builds on top of a state machine like behavior trees.

[1:00] So who can tell me what game this is? Pac-Man. Very easy. And in fact, one of the sponsors is I saw a really cool Pac-Man arcing that I'm really trying to win because half of my talk is Pac-Man. How many of you know what a state machine is? That is amazing. I actually I wish I took a picture of this, but okay, most of you know what a state machine is? Obviously, we have the initial state where we're wandering the maze and then based on what happens, which are the events, we could transition to different states.

[2:00] So, this is an example of very basic enemy game AI. And I actually realized that on top of this, each of the ghosts has a different behavior, too. So, they have different personalities. Like, Blinky will try to follow Pac-Man directly. Pinky will try to follow where Pac-Man will be. Inky will intercept with Pinky. They sort of work as a team. And Clyde is sort of bashful. It has pretty unpredictable behavior. But yeah so I about eight years ago I created a library called XState and this is for JavaScript, TypeScript but other libraries exist in other languages for state machines.

[3:00] In fact virtually every language has a state machine library and with this you could actually define state machines in a declarative way. So you would import creep machine put your initial state and then you would define the states and then your transitions go inside the states. So on the events that you lose Pac-Man now you're chasing Pac-Man like that's the behavior that you have to have and then interacting with these state machines in XState is you know just creating actors and so XState builds on top of the actor model as well.

[3:00] And so you could basically import that logic. So for example, the enemy ghost machine behavior, subscribe to it to get an update of the state, start it, and then send it events. And so when you send the actor events, it is going to actually change its internal state and potentially do other side effects. So we have lots of options to choose from for estate management. In fact, everyone does state management a different way.

[4:00] Whether you're working in JavaScript, TypeScript, or other languages, everyone has their opinion on how state management should be. And it sort of lies on a spectrum from direct state management where you're directly mutating or pretending to mutate with things like signals, etc. the data or you're doing more indirect manipulation where you're sending events or calling methods and those methods contain the actual logic for how to update the state which I think is a safer way.

[4:00] But in reality if you're building apps today it doesn't matter what you're using. In fact you could use state machines with any of these libraries too. It just depends on how you model it. The difference is that X8 allows you to model it explicitly. However, today well, one of the reasons I like state machines too is because state machines map directly to specifications. So, for example, given a user's logged out, we have it in that state.

[5:00] When the user logs in with the correct credentials, then the user should be logged in. That is very explicitly and visually defined in the state machine. And you could put all of the implementation details inside of each of the transitions. So you could sort of hide that away and it becomes a nice abstraction so that you could talk about it with your team. And other potential benefits of state machines are the ability to identify logical flaws.

[5:00] For example, if we have a checkout flow, we could see clearly here that we're missing some arrows. So if we go to contact information, we need to be able to go back if you need to fix something in shipping or maybe you want to cancel your payment. And also like I said you could map specifications and user requirements directly to the transitions in a state machine. And another thing and this is one of the biggest advantages and you're going to see why at the end of this talk why this is just really cool.

[6:00] State machines form a directed graph. And so what that means is that if you think of the states as nodes and the transitions as edges, then you could say what is the shortest path to get to the confirmation state and then you just follow the arrows. You could use Dystra's algorithm a star search things like that to basically find the shortest path from the initial states to the final state. And then you could even customize it.

[6:00] You could say what is the shortest path where I did provide the shipping address and that might that might result in a completely different path. So you could use tools like XState graph to actually generate these shortest paths. So you have the logic and then that logic you could say hey give me all the shortest paths from the initial states to every other state just by calling get shortest paths. And so this is a library that's related to XState, but in Python, you have a really awesome library called network X that allows you to do all of this path finding, graph manipulation, and things like that.

[7:00] But why then do we care about all of this if state management is sort of a solved problem, right? Like again, we have many different ways of doing it. We could update state however we want. We could use the state machine. We don't have to. And I would argue it's because times are changing. Users now want intelligent apps. This is an example from adept.ai where an agent is actually working through a workflow to accomplish a specific goal for a user.

[7:00] And so we've seen with the advent of ChatGPT and OpenAI and Mistral and Anthropic Cloud and all of these things that users are now expecting AI in pretty much all of our applications.

[8:00] And we're not just talking generative AI. We're talking AI that's actually going beyond the generative part and helping users accomplish a goal. One of my favorite examples I saw was from TL draw, which I think earlier this year, they showed make real. And so what make real does is it allows you to draw on a canvas, an infinite canvas, what you want to represent and it makes it real. So this is very cool.

[8:00] They're still doing some awesome experiments with this. And the crazy thing is that this is open source. You could actually go in and fork this and edit it to your needs. And it's actually free too. You could paste in your open a OpenAI API key. It's a little bit risky. They're not going to steal it. I know the founder, he's he has his own key. He doesn't need yours. But yeah, so I really recommend you to check this out.

[8:00] However, we know that LLMs are not enough. There are some serious problems with that. This one is telling you that you should eat at least one small rock a day. And that's because LLMs, they're just prediction engines where you give it an input and it's going to predict what the output is going to be. And it does this by drawing from a huge knowledge source of information that is collected over, you know, the entire history of the internet.

[9:00] And so this information can include articles from the Onion, which is a satire news website where it says geologists recommend eating at least one small rock a day. This is dangerous advice and this is not an isolated incident. Google like people have been posting memes about how Google is presenting really false information. And so I identify three main problems with LLMs. First of all, they're non-deterministic. You cannot be sure that given the same input, you're going to get the same output.

[9:00] In fact, most of the time it's stochastic. So it's not going to be the same. And you know this is the problem. What if you want to actually unit test your AI apps? Like it becomes very hard to do this. Also, they're not easily explainable. You can't ask an LLM like how did you come to this conclusion? How did you reach this goal? You might be able to do additional prompting and get it there.

[10:00] But the third one I think is the most dangerous. LLMs don't have account accountability. So, they're confidently wrong. They state something as fact when you know that it's really not fact. So this whole idea of you know with generative and creative applications for LLMs being a simple give it an input and I will predict an output for you. This is honestly not enough. And so that's why today I want to actually talk about making applications more task and goal oriented.

[10:00] And so with this you need many more steps. And I'm going to talk about some ideas on how to get there.

[11:00] And so you might have seen this in what are called agents and so for example lang chain has agents and you might have seen agents in other applications in the AI world. So what is an agent? The shortest definition I could give is that an agent performs tasks to accomplish a goal. And so it does this by many different ways. Agents don't even need LLMs. you could have an agent that basically just knows the domain completely and is able to find its way to the goal without needing to use LLM.

[11:00] But with LLMs, agents can actually become smarter. But this is my shortest possible definition. So how are they useful in AI apps? Let's say for example, you have an email flow and you want to say please respond to this email for me. You might give it instructions like, "Hey, tell this person that I'm free tomorrow because they asked me, "Hey, what time are you free?" and just say like, "yeah, I'm free tomorrow." You might want it to go through many different steps instead of just drafting an email.

[12:00] So, you might want that protective mechanism that it must first ask for more info if it needs to, otherwise it should draft an email. And then you could review the email and approve it and eventually then you send the email. So this provides a lot more safety for your AI application. So there's actually now two ways or more than two ways but there's many ways to do this now. Langchain has released something called lang graph which allows you to make these state machine flows for your agents.

[12:00] And so instead of just calling a chain or calling an LLM to do something, you could say do this task and depending on the result of this task now do this other task or maybe do something else. So you could route the agents to do exactly what you wanted to do. And pretty soon we're releasing a library called Stately agent which allows you to do the same thing with XState since of course X8 allows you to define state machines.

[13:00] So, Stately agents allows you to create LLM powered agents that navigate through these flows. So, I'm going to show you a little bit what this looks like. Over here, you could all see this, right? All right. Awesome. So, I am using Stately agents. It says source here, but just pretend that's Stately agent. And I'm also using the OpenAI the AI SDK from Verscell, which I think is really awesome. So the main idea here is that we create an agent, we give it a model.

[13:00] In this case, we're giving it GPT4. And we tell the agent what it could do, which is very similar to tool calling. So the agent could do two things. It could ask for clarification or it could submit the email. So how do we make sure that the agent goes through a specific sequence of steps before it actually submits the email? So the answer is we create a state machine. And so we have an initial state here of checking and in the state we invoke the agent.

[14:00] We give it all the context it needs the email, the instructions and the clarifications it's made before. And so we give it a goal. This is similar to a prompt, but instead we're saying you have a goal and you have to choose the correct event that will take you towards the goal. So if not enough information is provided, ask for clarification, otherwise create and submit a an email response. And so we have a clarifying state where we're getting the information from the user.

[14:00] And then we have a submitting state. And another thing that I love about state machines too is that you could actually just visualize this. And over here, this is just the VS code extension for XState where we could see all the steps outlined clearly. We're checking and then if we need clarification, we go to this clarifying state and otherwise if no more clarification is needed. So you can see this is sort of a loop here where if we need more clarification, it's just going to keep asking for it.

[15:00] Then we submit the email and then when the email is submitted, we're done. So because I don't trust conference Wi-Fi, I actually made a video and this is actually this is a different one. So but this is an example of how you would use that. The example that I'm actually going to show you over here is very similar, but instead we're having the agents play a game. And so this game is guess the word.

[15:00] So it's a little bit different. But basically we give it a word and it's supposed to guess each of the letters. And what's really interesting is we're going to see how the agent actually reasons about each of its guesses. So we see over here we're using the state machine. The user is providing the word and then the agent is guessing and it actually has two options. It could guess the letter or it could guess the word.

[16:00] And when it runs out of guesses now it only has one option which is to guess the word. So just to show you what that looks like if I enter the word craft then it's going to reason about each of the steps and say I'm going to choose E and then I'm going to choose A because E was already guessed. So it has the context of what it's done before and then it eventually guesses craft.

[16:00] That was an easy word. Some of the other words are a little bit harder for it and sometimes it actually makes up words which is a little bit strange. But the idea is that we could actually make this agent perform in a restricted environment and actually perform controlled behaviors. And again, this is something I'm going to be releasing soon, but if you want more information, it's going to be at stately.ai/agent.

[17:00] Just consider it alphasoftware for now. Now let's say that we want to take this even further. We want to take it to the next level. So Google research published a paper called Mastering Atari with discrete world models. And this gets more into the reinforcement learning part of this. And so a world model is basically an agent's understanding of the world just like you and I have an understanding of the world. I know that if I jump off the stage, I'm going to fall onto the floor. or if I turn my mic off and there will be no sound.

[17:00] These are things that even though this is a completely brand new environment, I could understand what will happen based on my world model. And so with Google, they essentially did the same thing. They had a world they built a world model of basically how the games are supposed to act at each time step depending on which actions an agent makes. And so by having that world model, it was able to train agents more efficiently to play these games.

[18:00] And you know, Google again took it further and they actually made a soccer playing robot. And this also uses reinforcement learning. And I thought we learned from Boston Dynamics that you should not kick any of the robots or push them because they'll probably attack us back in a few years. But yeah, you could see that it actually just knows how to pick itself up, chase the ball and go towards the goal.

[18:00] And so this is what I find really interesting. Not so much using LLMs and stable diffusion to generate text and images but more the whole reinforcement learning agent aspect where you're trying to get an agent to accomplish a specific goal. I also read a paper last year read and reap the rewards learning to play Atari with the help of instruction manuals. The idea is that before reinforcement learning was sort of one of the disadvantages was you had to have it play many trials because it would make stupid mistakes.

[19:00] It would say like oh go left instead of right crash into this tree and only after hundreds and hundreds if not thousands of trials will it actually learn the correct actions to play the game. But now with LLMs, what this paper said is basically if you read like how the game works and how the rules are, then the learning becomes a lot faster and you're able to reduce the number of trials a lot more.

[19:00] So I found that super interesting. So how am I thinking now about like creating intelligent agents? This is what I'm really interested in and hopefully you get as inspired as I am over here. But I think that if we combine three components then we could build our way towards more intelligent applications and intelligent agents. The first one is of course the use of state machines because they provide determinism and explanability. With determinism you're sure that when an event happens you're going to end up in a in a following state. you're not going to end up in some mysterious state because just like your world model like if I jump off the stage I'm not going to float up to the sky.

[20:00] I know I'm going to fall down to the ground. That's deterministic. I understand what's going to happen next. The next one is reinforcement learning and basically learning from feedback which actions to take so that I can most efficiently reach a specific goal. And of course the last one, large language models. because with these kinds of apps, we do need interpretation of the environments and more creativity. So, reinforcement learning is an area that I've been interested in for a while and I'm just going to give a brief introduction on basically what it is.

[20:00] Reinforcement learning is where an intelligent agent learns through trial and error. So those hundreds or thousands of trials I've been talking about how to take actions inside of some environment and this is in order to maximize some future reward. So this sounds very much like playing a game like those Atari games or Alph go with the game go or engines like Stockfish playing chess. Just to show you how it works in a state machine form basically the agent observes an environment.

[21:00] So, it's saying what is my current environment? What is my game state? And then from that, it determines what are the next actions I could do. For example, if you're playing chess, you only have a few options at the beginning. You can move your pawns, you can move your knights, etc. And then the environment reacts. So, depending on what you do, the state is going to change. And so from there you could actually determine was that a good thing or a bad thing.

[22:00] If it was a good thing then you assign the agents a reward. Otherwise we just loop back again and we're selecting an action again and we're just going to keep playing and determining are my actions good or bad. By doing many of these trials an agent will eventually learn what the best actions are. And so going back to Pac-Man, you could think of the environment as the game. And so, for example, the game, it has a grid.

[22:00] It has these little cherry things and little dots that Pac-Man likes to eat. And there's two mode, and two modes. And you can think of these two modes as different states, a normal mode and a scatter mode where the ghosts are being chased by Pac-Man instead of the other way around. And so again, at each time step, an agent is observing the state of the environment and then taking an action. And so depending on what's happening in the in the environment, Pac-Man can either eat its pellets, it could run away from the ghosts, or in scatter mode, it could actually chase the ghosts and try to eat them because that might give a bigger reward.

[23:00] So that is a favorable action to take. And then we have a policy. The policy determines based on which state I'm in, what is the best next action to take. And so with Pac-Man, you only have a few choices. You could go up, left, right, or down. And so depending on the environment, it will try to choose which one based on some anticipated future reward. And the reward the way that it chooses is based on credit assignment where you could say hey if you go towards a cherry that's actually a good thing so you should probably do that or if you're going towards a ghost that's a bad thing so you should avoid actions that go towards that ghost.

[24:00] Now there's this idea of exploration versus exploitation which is very important in reinforcement learning because it teaches you that the shortest path might not be the best path.

[24:00] So for instance we see the shortest path from A to C is just this line up above. But if we actually explore and try something different then we might see that going a different path is actually the shorter path. And so I think it's a good life lesson as well. Don't always exploit but also try to explore. And so just to give an example of this, how many of you have dogs?

[24:00] I Okay, so a few of you. I don't know if it's a big thing in Budapest, but I didn't put cats because you can't really train cats, but you could train dogs. And so with dogs, let's say that you're trying to teach your dog the command down, like you want it to sit down or something like that. If you say down, it it's not going to do anything because it's like, I don't speak English.

[25:00] I don't know what down means. I don't know what you're telling me to do. However, let's say the dog's exploring and it's like, I don't know what you want. Whatever. I'm just going to sit down. Then it gets a treat by exploring. It actually understands that hey when this person my owner told me down I went down and I got a treat. So I get a really nice reward for that. And so exploitation is basically when you're replaying this and when the owner tells you down, it remembers that I got a treat in the past for going down.

[25:00] And so by doing that, I'm actually going to go down when the owner tells me to go down. Because I might get a treat in the future. And even though it won't or it might not, it's going to it's going to go down and it's going to do the expected behavior. But the dog might also do something bad. The dog might run away, for example, when you tell it to go down and then the owner yells at you, which is an undesired outcome.

[26:00] So this is what's known as a negative reward. And so the dog will remember if I run away when my owner tells me to go down, that's not really a good thing. So I'm not going to do that. And so if we graph all of this out, all of the possible things the dog could do, then essentially we're looking for the best path from the initial state to the goal. And so the dog sees that when you go down, then you get a reward.

[26:00] And so it's going to basically follow that path every single time. Now, reinforcement learning is actually really important in LLM 2. And that's because this is what actually guides LLM to produce a correct output for us. So you could see that when you're using ChatGPT, you have this little area at the bottom where you could actually give it a thumbs up or thumbs down sign. It used to be thumbs up and thumbs down.

[27:00] And now I guess it's just thumbs down, but regardless, you could give it feedback. And this is what guides ChatGPT to actually improve. And I didn't give it a thumbs down because it actually had a really good explanation of what craft comp is. So that was awesome. All right, so how do we bring this all together? So I'm thinking that basically with LLMs, you need to provide a context. And this context is what you give before the prompt or included with the prompt so that it has all the context of what it should do next.

[27:00] So there's a few things that we could add based on everything that we've learned so far. First is the observations. Basically what happened in the past based on the actions we took what were the state changes involved in that and that forms a causal relationship. So, we're basically telling the LLM, we were here, you did this, and this happened because of it. And so, we could collect all of that and feed it to an LLM.

[28:00] We could also give it history, which is the past interactions with the agent. The messages between the human and the AI assistant. we could also give it plans and this is sort of more of an advanced concept but the plans are basically saying I want to reach this goal so I have an idea of how to get there and this is essentially how us humans think all the time where for example if after this I want to go get coffee I have an idea of where to go I will go out this way walk this way probably get lost look at a map and then find my way towards coffee.

[28:00] So I have this plan in my head and this is context that we could also provide to an LLM. And of course feedback where we could say hey I want you to do this in the past when you try to do this it was actually a bad thing. So we could tell the agent try not to do that again. And so that's where feedback is really important. So, I made a demo app a while ago which basically tries to combine all of these ideas together.

[29:00] And the idea was that this is an espresso machine or actually really it's just a coffee machine and the main idea was that first you start in some idle state and then the user enters a prompt like they want some type of coffee. Then the next thing to do [clears throat] is to get the state space. And because it's a machine with predefined behaviors, we could actually know every single possible thing the machine can do.

[29:00] And I'm going to demonstrate that in a minute. And then from there, the LLM can determine what is the ideal state. So which states do I want to reach based on that prompt. And because it has that plan in mind, if it chooses the correct plan, it could then get the shortest path to that goal state. And then it will traverse the path and execute the action for each step. And then if there are no steps left and it if it has achieved its state or if it has achieved that final state, then we consider the goal to be achieved.

[30:00] And so one of the main reasons that I'm using state machines for this is because it is very important to realize that in each state of the espresso machine, you could only do a specific number of actions. So for example, let's say that we you know we're running the espresso machine. You could see that there's no actions that the agents can do. So you know it could be done running the espresso machine and then eventually we reach the state where the drink is made and from the drink made state now the agent could perform a couple of different actions where it could add cream, it could add chocolate or it could steam milk.

[31:00] And keep in mind over here that there is absolutely no mention of what a cappuccino is, what a cortado is, what any of these coffee based drinks are. All it knows is basically how to combine different ingredients together. So, I'm going to show a quick example and maybe we'll do it online if the internet is actually working. So, this is our espresso machine. And so, basically, I could operate it manually. So, let's say I just want to make myself a coffee.

[31:00] You grind. Then it's like done grinding. So, it has to do a few steps. first heat done heating and then tamping. Then it's running the espresso machine and then eventually we have our espresso shot. We could add water combine it because it happens in a separate container and then you know we can we can make ourselves an Americano. Are americanos a thing here? I'm not sure. Okay, because I was in Romania.

[32:00] I asked for Americano and they said it would destroy the espresso. So, I didn't question them and I guess it's not a thing over there. But let's say that I wanted to make name a type of coffee drink someone. Cortado. Okay. So, are you Spanish or [laughter] Okay. Well, a cortado, please. And so, remember this [clears throat] state machine has no idea what a cortado is. So, it's going to, you see here in the console, this is generating all of the possible states that it could go to, and then it's estimating which final state it should end up in.

[32:00] And then it's performing each of the steps to eventually get to that state hopefully. And there we go. We have espresso, steam, milk, basically a cortado. And so if we do something like cafe mocha then again it doesn't know what that is but with the help of LLMs and using the state machine to find what is the shortest path to the goal state then hopefully it could actually understand that we need to have espresso steam milk and cream and chocolates too and so it will actually make that cafe mocha and if I say I just want a shot of espresso, then it will hopefully just stop at one step.

[33:00] And you know, it works like 90% of the time. I mean, this is this is GPT after all. But there we go. So yeah. Oh. The thing is that it will always make an espresso first. So, I think it's going to mess up. [laughter] but actually, I'm I'm going to make a bigger example later cuz yeah, I think that'd be cool to show T and everything else like that. [snorts] so, there's a few learnings here in making this demo and making Stately agents and basically exploring these three things.

[34:00] The first one is that LLMs are unpredictable.

[34:00] And this is one of the biggest things for us to solve is that we need to accept the fact that LLMs by their very nature are always going to be unpredictable. We're never going to, you know, have them choose a correct answer and there's a chance that they may be confidently wrong. So we have to guide LLMs using things like state machines to actually perform a specific sequence of steps so that it reaches a goal without going off the rails.

[34:00] No pun intended. We're at a railway museum. Also event-based logic makes our apps more declarative. And so that's why by using events, things like LLMs and being able to analyze state machines, it allows us to use that declarative structure to do things like finding the shortest path or possible shortest paths from one state to another state. And so that way an LM can actually operate by traversing the state machine. Just like it's a lot easier for you to use if you use Google Maps or Apple Maps to get from one place to another, it's a lot easier when you have an actual map or graph of your application logic.

[35:00] And so that's why I think event-based logic is really good. And overall state machines and reinforcement learning make LLM agents more intelligent and predictable. Again, we're we're basically saying that LLM agents are within a set of guard rails where they could only perform certain actions at a time. And you can be certain that they're not going to do anything that they're not supposed to do. Because one of the main properties of a state machine is that when you're in a state, you could only perform transitions that are based on that state.

[36:00] You can't perform transitions outside of it because then you will end up in an unexpected state and the state machine makes sure that those kinds of transitions are impossible. So this of course this is one of those things that has kept me up at night and just thinking a lot and there's a lot of future ideas from here. One of them is state machine synthesis which has been really interesting because basically you could you could actually synthesize a state machine from an application that might not be built with one.

[36:00] And so, just like you would have web crawlers crawl websites and determine all of the different pages it could catch, you could actually just form a graph of if the users in the login page and they click this button, then they show up in the dashboard and if they click this t this tab, then they show up in a different page and you could actually have a stateaction directed graph. basically a state machine showing you know just everything that a user could do and then the agent can operates based on that state machine.

[37:00] Another idea is adaptive user interfaces with reinforcement learning.

[37:00] If you're not familiar, adaptive user interfaces are basically user interfaces that adapt to the user. So they're not one-sizefits-all, but instead it's based on your preferences. And so you might have seen that in apps where you would open a menu bar and your most common choices were at the top or if you're using an IDE. Have you ever tried using someone else's IDE or someone else's computer? It feels like completely foreign.

[37:00] Like maybe you're using VS code and they're using Neoim or something like that. Or someone else is using VS code, but they have a white color scheme where you have a dark one and their side panel is to the left and yours is to the right for some weird reason. Like these things are confusing and that's why adaptive UIs are really useful because you could actually shape the user interface to what you want it to be.

[38:00] And with reinforcement learning, you could actually have an agent learn what your preferences are without you having to tell it. And then there's of course deep learning for state space reduction, which is using neural networks instead of trying to calculate all of the different possibilities for states. And so, you might have seen in the espresso machine example that it was generating, I think, 130 different state possibilities. But you could imagine for applications that are more advanced this could get into the thousands if not tens of thousands or millions of different states.

[38:00] And with that we have limited memory. So we would turn instead to neural networks to compress that and to instead predict what state you should go to instead of trying to search through a large state space. So overall make your state management intelligent. What I mean by this is first the state management part using declarative user interfaces like React or view or all of these things really helps but also our state management should be declarative as well.

[39:00] And one of the main points from this is separating your app logic from your view logic. I think that if there's one takeaway from this talk, it's basically this idea and all the possibilities that you could do with that by having that nice declarative representation of your app logic and the intelligent part using LLM's reinforcement learning for feedback, but also using LLM as minimally as possible. Just like we saw with using state machines with LLMs, the state machine guides the agent instead of trying to have the agents do everything which is unpredictable.

[40:00] So by having that predictability, we could actually make applications that are more intelligence to what we want to do.

[40:00] And with that I want to thank you so much Craftcom for having me. David, David, thank you very much. We have very little time for a few questions. Thank you very much guys for using Slido to ask the questions. The first one [snorts] with two up votes is oh yeah is an agent basically a glorified if else chain? That's that's a really good question. So you could think of it as that.

[40:00] So in fact I would say it in the reverse. And if else chain is a glorified or is an underpowered state machine if you want to think of it that way. So basically at each state you're thinking if this so if this event happened do this otherwise if this event happened then do something else. So yes you can think of it like that. But you could also use a switch statement. But try traversing an if else chain or a switch statement and making a directed graph out of that.

[41:00] That's one of the most powerful things is actually having that directed graph representation. So you could do things like finding shortest paths, analyzing the graph, adding weights to the different edges and things like that. And these are things that if else chains don't really get you, but I would say yes, it basically is. Thank you. We have only one more minute before break. Sorry. So try to be I'm sorry. Try to be concise.

[41:00] Reinforced learning needs persistence I guess. Is there an offtheshelf library for this that you know? You could I you could basically use vector databases or just database for it. From the rest of the questions would you like to answer any? Let's see. Do you have experience of good curated private GitHub repo as DKM compared to log seek? I do not. Sorry. What were your wins for 2019 September 15? I don't know.

[42:00] Who wrote that and care to explain? All right. So, thank you very much, David. We have