Reactive State Machines and Statecharts (React Finland 2018)

For me, my name is David Khourshid. And I'm gonna be talking about reactive state machines and statecharts, which follows the last two talks. Pretty perfectly. So, show of hands. How many of you are using state machines or statecharts, you know, in your React applications or any of your applications? All right, there's literally like, five hands up. Let's try this again. How many of you are using these variables? Come on, all of you shame on you, you should not be using these variables. We talked about making impossible states impossible, and these are the two biggest culprits for, you know, doing that. But user interfaces are hard, and they're especially hard to develop in the United states. We had a little situation, you know, where we accidentally sent a missile warning in Hawaii and that caused a huge, huge disturbance in Hawaii. But the thing is, we see this pattern all the time. In fact, I see it on Twitter like I tried to click my own name and I went to Donald Trump in it.

You know, it just did not work. So we realized that the most neglected variable when we're doing development is time, and indeed, we have so many different solutions for how we deal with time. You know, we have callbacks, we have promises, we have a sink, a wait. And it might my favorite pattern, honestly, we have observed a balls. And so we have so many different ways of dealing with time. And each of those ways are different in their own respect, and there's probably many, many other different ways the. The problem is that we need to have some sort of common way to model the way we deal with time-based events in our user interfaces.

Play from 0:00

But we have a state management libraries. I love this quote. It's any sufficiently complicated model class contains basically half of a state machine. It's like it's almost there, and if you don't know what the state machine is, that's okay. We're going to be talking about it a little bit, but I'm gonna replace complicated model class with state management library. Because we have Redux, we have Mob X, we have, Unstated, we have. I don't know, I'm sure, can we, Laurel, invent something? But all of these are great, you know, in their own respects, but. They're, they're not exactly modeling our application in a way where we could fully know every single possible state and every single way we could go from state to state. And that's what we're gonna talk about today. So what I want to present to you, even though this is called Reacts, Finland is a framework.

Lissa Proach because I believe that some of the most innovative libraries that come today, whether it's in JavaScript or in reason, or any other language, our framework list. They're not React specific, they're not view or Angular specific. So things like Redux, Earth Mob, XAR, Lodash, or, you know, all of these different things don't rely on frameworks. And that's what we're gonna try to do today. So the question is, how can we model the behavior of user interfaces? And by heap behavior, I mean how users interact with the system and how they do so over time. When we think of an API application programming interface, we think of something that is documented, we think of something that is predictable, and something that's also testable. However, I think that U.S. front-end developers, or whether you do friend and their back-end.

Play from 2:00

This still remains true that the most difficult part of it is dealing with humans, and humans are undocumented. I mean, we're, we're still try and discover how we work, we are very unpredictable and we can't really be tested. I mean, that's that's illegal after what Germany did anyway, so let's, it's a bad joke. Let's do an example, and this is, this isn't reacts. But you could imagine that this would be in view or Angular, or any other framework, or even no framework at all by them using set state, so we're using React. So let's say that we wanted to create a simple search app where we search for images on Flickr. So, you know, it seems pretty easy. We pass in a query and we get the I have a laser. So we use a Flickr API to query the data. And then, once the promise resolves, we just set state the data, and then we showed the data when the results are retrieved. So what is wrong with this?

Well, I mean, the code works perfectly, it does exactly what we want to, but what if we're on the slow network connection, okay? So now we have to show a loading screen and we're gonna be using that terrible variable that I talked about, the loading variable. And so now when we get the data, we need to set that loading variable to false, and then we have to set the data. Okay, now our apps perfect, right? What if the API fails, right? Then we have to show an error screen? And so now we have a different error variable and we set that flag to true and so we set loading to false. Because we don't want to show the spinner and show an error at the same time, that's when it'd be impossible, states, right? And so a good user interface doesn't like show a bunch of loading things and errors at the same time.

Microsoft does, though, so there's a problem here, though. When we want to retry and search again, then the error state is still showing, so we have to set that to false. And then we have to do a few other things too. We have to hide the loading screen, show the data, hide the error. But if you know if an error happens, then you know, we have to just make sure we're managing our state correctly. It's getting really, really complicated right now. So, now let's say that we get another request. We wants to cancel the search, so the search is in flight, and then we have a cancel button because maybe the search is taking too long. Okay, so this becomes a problem. We have this on cancel method, which has to set loading to false, air to false and cancel to true. So we have this little canceled tag.

And then, in our previous error handler or sorry search handler, we need to set that cancel to false, but we also have to check if it's cancelled.

Play from 4:00

Actually, I don't even think it's up there. See, I already have a bug in my code and I don't know if you could read this. Let me zoom in for you. This is a mess, honestly. Our event handlers got really, really big and we need to deal with that. This is the way that we all program. In fact, I'm very guilty. I've been doing this for the last, you know, six years. It's called the bottom-up approach. And the way it works is that we have an event, you know, event happens anywhere, think of it as on click or on change. And those events do certain actions, like it's set state there, it does a fetch call there, anything, and all of those actions affect the state in many different places. This is really, really hard to track because bottom-up code is very difficult to test.

I mean, if something is just imperatively calling in action, how do you know that action is about to be called? You have to intercept it or use a spy or something. It's also difficult to understand you can't just look at the code and understand the application logic of what the code is doing. You have to rely on the developer telling you, Oh, this is how it's supposed to work, and it might or might not work that way. The code will contain bugs. And this is almost a guarantee, just because there's unforeseen edge cases and states that we don't know about. And that they're only going to be discovered by a user who's using your app in some sort of weird way. It's also very difficult to enhance. Where it's like, you have this nice, complicated mess of code inside your event handler. And then your project manager says, I want to add this feature, and then you're like, Ah, I don't want to add that feature because my code will be terrible.

And you know, these features make it worse, they make the number of bugs that are possible in your system grow exponentially higher. So here's some intuition. First of all, user interface components are not independent. I mean, we tend to think of them as like, Okay, let's make them self contain an independent. But with libraries like Redux and Mob X and RxJS, and things like React context and things like set state and whatever, we know that this is not true. We know that our components are going to send events somewhere, and those events are going to affect the state, and that state is going to affect other components too. So there's this implicit connection between components. Also, we know that actions are based on events and the current state like a counter example, you know, you press increment, your action isn't going to set it to two. Or set it to three, it has to know what is the current value, add one to that, and then set the state.

But also this event action paradigm is too simple. Because once we get some more advanced logic cases. Like, let's say you have a simple counter app, but you want something that says the user must. You know, the user cannot be allowed to count more than 10, or the users accounts doesn't have permissions to increment the counter. All of these logic things go inside your event handlers and just make your code a lot more complex. So this is where finite state machines and statecharts come in. If you were to look this up on Wikipedia, or maybe you took a university course event, I didn't, thankfully. But apparently it's a very, very boring, hard subject. But I'm going to try to explain it visually. And this is a part that I love about state machines is that it's a very visual thing. So finite state machines have one initial states, because you have to start somewhere, right?

They also have a finite number of states. So in this case, like a promise implementation, you could either be in the pending or fulfilled or rejected state. You start from idle, so you can't be both in fulfilled and rejected at the same time. It's impossible. It also has a finite number of events that could happen. For example, we can make a fetch request and then that fetch request can send a resolved event or a reject events. And those transition to the proper states. And also there is a finite number of final states, which we're not going to talk about here because it doesn't exactly apply. So, in our sample app, the one that, you know, I showed the big hole code mess up. Here's how we could represent that as a state machine. So we start in the idle state and then once we start searching, we get to this search state.

And from there, you know, two things can happen. We can either resolve or reject that promise, but I'm adding a feature here, or we could actually search again. And this is important because in our gallery app, we want to be a more than one time use. And so you can see that those clearly transition back. From whatever state we have to the searching state, and another important part is right here, this is called a self transition. So that if we're in the searching state and we keep clicking the search button, then we could say, listen, were already in the search state. Don't try to make multiple API calls. Alright, so remember that problem I was talking about earlier - this is one possible solution to it, you know, modeling it has a finite state machine. I was thinking that, you know, if you're on the first date where something is showing and then something changes like the data is updated.

Waits like half a second or so, and don't let the user touch anything until those three seconds or 0.3 seconds are up. And that way, because we don't have a click handler on this intermediate state, now, the user won't accidentally click something. I mean, this is the simplest solution to a user interface problem that we see all the time. All right, so how do these look? If you were to, you know, look in old textbooks, or, you know, other implementations. You might see a state machine as a switch statement. In fact, you might be writing state machines without even knowing it, and actually, you definitely are. These switch statements. They're good, they're, I don't know. Fast performance. However, you wants to micro optimize, but you can't really enumerate through each of the states or each of the actions. And that's why I like using an object, and so this object, it sort of looks like a switch statement.

Where we ask, what, what state am I on? For example, searching and then what events just happened, such as resolve, and then it will give me the next state, so if I'm on searching and I gather resolved events. Then we're in the success state and then I have a very small transition function that does exactly that. It takes the current state, looks it up from machine dot states. And then we go to the on property and we figure out what the next state is from there. So, here's an example of that structure, the load. So this is exactly the app that we are trying to model. So if I search for a Finland, you can see that currently we're on the start state. And it's going to the loading seat. But I could press cancel, and so I don't have any fancy JavaScript or any fancy logic for handling cancel, I just have. If I go to cancel search, then I'm in this gallery view.

And so even if I get a results back from the server because I'm in this gallery view, the gallery doesn't handle that. Hey, I have some pictures, results, so the gallery just says, yeah, I don't care. But now if we search and we let the search go through, then we get these photos of Finland. I think I don't know what's that. So, and then, you know, of course, you could search again. Oh, and here's, here's a good thing, like, the button is disabled right now. But you know, sometimes in your app, a user can jam a button so many times, and then that causes many API calls. But in this loading state, we don't have a transition from loading to search. So they could click as many times as they want, they could remove that disabled attribute. It's not going to do anything, which is exactly what we want.

So I work at Microsoft and I work on the visual Studio live CHER team, and I decided that I wanted to experiment with this very complex part of our user interface. If you don't know what live share is, first of all, you go to this link to check it out. But live share is a way of collaboratively coding between pretty much two people, anywhere in the world, whether you have visual studio, code or visual studio. But we have a pretty complex sign-in flow, so at first you're signed out and then you know, you sign in, it's signing in. And in fact, there's more states than this. Because we could automatically sign in, or we could search for our code. And so if the sign is sign in is a success, then we could go to either this sharing or joining state. If you're sharing, it means that you are sharing your code with someone and someone else can go in and edit it.

You know, from the comfort of their own IDE. And of course, these could have, like arrow states, too, and so we could only be in one of these dates at a time, and this is. Important for showing like the right message in the status bar. So by doing this, you know, we removed a whole class of bugs and user interface like glitches and errors. And I was also able to use that exact state machine for analytics. Because one important thing at Microsoft is how users use our applications. And so we use telemetry for that, and I'm sure many of you do as well. And so in this simple transition function, I just have this telemetry that sends date and we send the current state, the next state and the event. And from that, I was able to automatically visuals on how users use our app. So we see that from this initializing state.

A lot of people go to the signing in state and then most of the time we see that they go to signed in, and sometimes that fails. But there's no error. State, it's somewhere, I promise. And then we could see whether the users are sharing more or joining more, and how that flow goes for them. Why this is important is because by having this what's called a weighted, directed acyclic graph, we could see which paths are the most taken. So, for example, we have from A to B and B to C. And then by visually saying that, we could optimize and say, what if we just made a direct connection from A to C? Because we could see that our users wants to go directly from A to C? And so that's one example of how you could use these visualizations, we could also use them for integration testing as well. Imagine, like, automatically generating all of your tests.

And so, what I would use is an algorithm, either a simple path or a shortest path algorithm. Where we just Traverse each of the states and we generate all of these different paths from each of our states. In our application, analytics can provide the weights so we can know which of these paths are the most frequently taken and the shortest path algorithm. It sort of represents the happy path, because users are going to go from one state to the end state as quickly as possible. In, we know that, for the most part, this is going to be the most commonly taken path, and of course, these could be. Automatically generated. There's also something called the simple path algorithm, and this is using a death first search algorithm. And so instead of going, you know, through the shortest path, it's just going to find all of the paths without cycles. And so, for instance, this would be one of those paths going from A to D to E to B to C.

And so it's not going to, you know, go repeat any states. And so this represents all of the possible user flows, as long as the user doesn't get themself inside of a cycle, which I actually did. I came here to the airport and you, you have to buy a ticket from a mobile app to use the train. And it wouldn't accept my credit card. But I was running on two hours of sleep, so I tried like five different times. And so I was getting this cycle if it just did not accept my credit card. And so, I mean, it is possible to get stuck in that cycle, but for the most part, that's not gonna happen. It also reveals all the different edge cases that could take. Because we're representing every single possible path that the user could take, and they could all be automatically generated. Now. A word of caution by doing these, you know, generating these simple paths, you will get a lot of paths, which is either a good thing or a bad thing.

It's a good thing because you could generate so many integration tests, it's a bad thing because you're going to generate so many integration tests. And they're going to run for, like, I don't know, 15, 20 minutes or an hour, depending on how your machine is set up. On the workshop that I gave the other day, we actually did this. We took an existing app, we wrote the state machine just like this. And we were able to generate 27 tests. And these are full integration tests, not unit tests. Automatically. From this date, restate machine definition. So, the reason I talk about testing so much is because most of the time in our companies we do this. We manually test the code, or we hire people to test the code. And we know that we could do better than this, we could write tons and tons of tests, right?

But? We could also model the code and automatically generate all of the possible tests that could happen based on that model of the code. And this is good because when we change, you know, the state machine or statechart, these tests will be automatically regenerated. Have you ever had to add a feature to an application and then hundreds of your tests broke? I mean, this solves that, if you're wondering, the next level after. This is called formal methods, like either using C, TL, A plus or Isabella, like just so many different ways to do formal methods. Where you could mathematically prove that your code has no bucks, and some people do this, it's not a very common thing, but it's important for mission critical systems. And then, of course, the most important way to test your code is to just delete it because it will not contain any bugs.

So software bucks are made visually clear with state machines, for instance, if we have this idle loading success states in an application, we could easily look at it and say, hold on. What happens if we have an error? We don't have any transition from that, so we add one, right? But then we look at this, this visualization again, and we see, okay, when we have an error, the user stuck in the failure state and there's no outgoing transitions from there.

Play from 7:00

So, of course, we need to add a retry and, you know, have some sort of retry button. And these are all extremely clear. So the question could be, does this scale? You might be thinking, I have a large application, you know, this might not exactly work, but this is where statecharts come in and they were. They were conceived by David Harel in the paper in 1987. So very, very long time ago, before I was born. And it's based on this idea of extending, you know what finite state machines are. And so it adds a few things that are extremely useful to applications today. First is actions. Of course, I know some of you were thinking, well, how do these state machines actually do anything? We have three different types of actions on entry, which is when you enter a state, on exit, when you leave the state, and transition actions when you go between states. We. Also have guards, and you could think of this as like form validation or things like that.

Where, for example, you could only search if you actually typed something into search, that would be an example of a guard or a conditional transition. There's also the idea of hierarchy. So instead of going from, say, A to B to C, like you would in a router, your, you know, your single routes could have many different steps. Look like a multi-step checkout form. There's also the idea of orthogonal D, or representing more than one state at the same time. And so, uh? For an example of this, consider an application where you have, like a little text editor. You can have bold italic underline, and you could click. You know, each of those representing all of those as like, simple states becomes exponentially, you know, difficult. Because, you could have everything off, or you could have everything on, or you could have bold on and everything else off, and so on, and so forth.

And you know, God help you. If you have like Strikethrough, ER and the other thing, then it becomes really, really hard to model. With doing it using orthogonal or parallel states, it just becomes this. And they're supposed to be dotted lines between each of those, so just imagine there are. But yeah, we could transition from bold on to fold off, italics onto italics, and underline on to underline off, and represent those as parallel regions. Also on, the notion of hierarchy can be used to simplify state machines, too, because it becomes very easy for finite state machines to quickly explode. In the number of states and number of transitions it could have, for example, these success and failure states. We have the common feature that we could both search from them. So if we group them into this search state, then we could group common behavior and be able to transition from those states over to the parent state.

Now I created a library called XState about I release it about 6 months ago and it does exactly this. It handles state machine, statecharts, and it's just a function. This is the most important part. We have our state machine definition here, the one we talked about, and the one where you don't even need to. Use a library to use. But it provides a helpful transit method where you give it the current state, the next events in the transitions to the next state. Can't really see that well, but it will give you like a value prop, and that value will be the next state. It does handle actions as well, guards hierarchy. So it's just a recursive structure and I don't expect you to read the code, which is why I'm like breezing through it. And also orthogonality, or parallel states, so representing states that could happen at the same time. And it also has the concept of history, too, which is part of the statechart definition, which means if you want to model like undo, redo, or going back, or going forward, I mean, that's already built into what statecharts are doing this and React.

You know, I taught a workshop on this the other day. And we could do this with just set state, we don't need to use anything fancy, we could be you set states. To first use the machine and figure out what our next state is. And from that we get are all of our actions. So first we set what the next state is, and then we call each of our actions. In each of those actions could update the state. And so we were able to make applications using this and use that statecharts to not only model our state management, but also visualize and test our application as well. And then there are event handlers, just become this send, so just like a Redux dispatch, we would call send, and that's all that would ever happen. No more logic in your event handlers. Now, of course, you could use this in RxJS as well, which is what the scan operator is really good for, or you could use it in Redux.

And the reason that you could use it in all of these things is because this machine thought transition function is just a pure reducer function, so any word that you would use a reducer, you could just use this machine dot transition function. Because it takes the current state and the next state. Here's another, quickly. Here's another app that I made where I'm trying to model like some complex behavior, so, for example, if I'm dragging. And I let go. Of course, it selects nothing, but if I'm dragging and I move over things, it selects. Some days. I could take these days and I could throw them away if I hover over the trash can. And then I could make my Sundays really, really long, and so I could edit my weeks. I wish this app really existed, but from this, and I'll show you right over here, those are just actions.

This is the statechart that I'm talking about. And so it pretty much looks like just a plain JSON structure. And this controls not only the entire app. But I literally copy and pasted this, put it in the visualization tool, and I generated this. So if I needed to explain to someone how my app works, I just show them this automatically generated from the code. We see we go from start to selecting to selected. And then we could drag, you know, the state of those days to make them longer, and then, you know, throw them away and all of that.

Play from 23:00

So, I already told you many advantages of using statecharts, but I don't want to be the ones to tell you. I want NASA to be the one to tell you. So these advantages, they come from an article that I read called statechart auto coding for the Curiosity Rover. Listen, we develop applications where we could test them ourselves, right? And we have QA testers and we could put them in some. Some environments where users can't see them, but we could have other users play around with it and eventually figure out all the bugs. When you're trying to trying to send a rover over to Mars, you can't exactly do that. NASA doesn't have an infinite amount of money. Or they could just keep sending Rovers, and if one of them breaks, they're like, Oh, okay, we'll figure out why that broke, and then we'll just send another one. Each one of these are hundreds of millions of dollars, or I think maybe, even, you know, a billion or so.

And so what they found was that you have visualized modeling, which is those visualizations. I was talking about very precise diagrams that you could hand off to not just developers, but designers and testers. Automatic code generation, which I know the big. Fan of? But they use? You could use those date charts and generate they generated, I think, C code or C++ code. Comprehensive test coverage, whether it's through automating tests or showing these state machine diagrams to testers. And then they could go and find all the possible states and accommodation of late-breaking requirements, which they found that they had to do a lot. But by looking at the statechart diagram, they were able to see how it will affect certain states and if it's even feasible. So there are certain disadvantages to. I'm not just going to talk about the advantages, but it's a high learning curve. Of course, modeling requires planning ahead, so when you're creating an application, you can't just start coding at the very beginning.

You have to think with pencil and paper. How is this all going to work? And not everything can be exactly modeled? Yet, you know, you might have some advanced use cases where it's like, how do I fit this into a statechart? But the good thing is that you don't need to use this day chart for everything. Maybe just your higher level app. States, but these are all trade offs, though using state machines. I mean, they, they make things simpler at first, but, of course, the number of possible states and transitions can explode. But with statecharts, you have higher complexity at first, but as requirements and features change, then you see that stay chart starts to level out. And they don't grow in complexity nearly as fast as doing things the normal bottom-up way, or with just finite state machines. So I talked about visualization, and I want to show you know all of you for the first time, this visualization tool that I made using.

React, of course. And so if you visit this, which I think I have it, open over here. That statechart code, the one that you Poe, you put into the machine, you copy, pace up there, press save and it's going to generate a diagram for you, and not just a diagram in interactive diagram. So let's say I'm in the start state and I click search. I don't care about the definition, it takes me to loading. I could go to success and then I could see what happens if I select a photo, OK, I. Go here exit photo. And I could also reset the chart. And of course, these are also draggable because, you know, sometimes they overlap and you want to make it look nice. So again, that's available on bit dot, lead Slash, XState - Vis if you want to check down, and of course, maybe you want to know, is XState being used in production?

Play from 30:00

I mean, I just released this six months ago. It just has a fun project where it's like it would be cool to think about how we model our applications. But it turns out that workshop taught me by Brian, Florence and others. They use XState for modeling their checkout flow, because apparently it's a really complex checkout flow. I don't know why we can't just give them our money, but apparently we have to do a lot of things in order to do it. And also Gatsby, maybe you've heard a bit. Static site generator, part of their processes now use XState in order to track. I guess this is code regeneration that's using Choke Adar. I'm not too familiar with Choke Adar, but it was still really cool to see that they were using XState as well. And we have a spectrum community, how many of you know of spectrum? Chat by Max Dover really, really awesome community.

I encourage you to check it out. But someone said that they just pushed their first couple of statecharts to production. And the logic was growing more and more confusing. Especially as you add features or you have to tweak things. But rewriting everything with statecharts made things so much clearer, especially since you could just visualize it instantly. So, of the future of XState. I want to eventually get full SC XML, which is a very old spec for this Gras era, not that it's ten years old, which, you know, in technology is. But it describes statecharts for any language, so imagine being able to share, you know, these states are definitions with C or Java or whatever. And reason to also a reactive interpreter. I want to make editable visualization tools, and, you know, more analysis and testing tools. And I want to do this in reason, so eventually I have to learn reason first, but apparently it's pretty simple. Thanks to Nick and. I also just released X8 3.2 now, if you haven't used X8, none of these matter to you.

But there's a whole bunch of nice improvements in the newest version of XState. And I'm going to be writing a blog post on this soon. So there's a bunch of resources, tools. One of my favorites is the world of statecharts, which is just a chart stock GitHub by Eric Mogensen, and it describes using XState. Just like why you would want to use statecharts and all of the different the different aspects of statecharts that you could use in your applications. So we first thought of write-once, right, anywhere, right, then learn what's right anywhere. That's what React introduced. But now I want to challenge you to model once, implement anywhere. It doesn't matter whether you're using React, View, Angular, no framework, Swift, Scotland, it doesn't matter. So let's improve the way we develop. Thank you so much, React in London [Applause].

Play from 34:00