Some Of The Things You Didn't Want To Know About State Tree In UE5 And Weren't Afraid To Ask

UPDATE 02/10/2024 - the Unreal 5.5 preview has updated the State Tree system a lot, which means some of the bugs / headaches / perceived miseries listed below may not be there any more- but if you’re not ready to make the jump to 5.5, the solutions and tips will still work.

The interface is also much clearer, and there are some new features. Particularly nice is the option to select states by evaluating a utility function - that is, rather than having to say explicitly where you’d like execution to go after a particular state completes, you can assign and modify weights that can be used to determine which state is currently most important, and then go to that. So, for example, you can assign a weight to a ‘fight’ task that scales with character health, and a weight to the ‘flee’ task that is inversely proportional to it.

This was something I’d implemented myself separately, but it’s now available right out of the box.

Handy!

But also slightly annoying given I’d spent a day writing a small, nicely templated library for selecting items probability distributions. That’s life.

ORIGINAL POST BELOW

I’ve been working with Unreal Engine 5 for a little while now, in between various contracts that pay actual money. The previous game idea is on the back-burner for now, partly because it’d require too much content to actually be good, and partly because VR is a tiny market. So, now I’m working on something more conventional, provisionally entitled The Last Gig.

Whilst UE5 is mostly a superb piece of software, it does have some niggles. The C++ macros, for example- these mostly let you forget about memory management and write things to interact nicely with the editor, but also hide a ton of complexity. That complexity is definitely better off hidden, but occasionally it will poke through and hit you with an error message even less comprehensible than the usual C++ error messages are.

In general though, that side is fine once you get used to it. But I don’t want to talk about that, I want to talk about the fancy new AI framework - State Tree - and some of the headaches I’ve had when using it. Some of these pain-points are simply due to how they work, but some are genuine bugs that I’ve had to work around.

It’s not all bad though! On balance I like State Trees, and so I’ve included some handy tips at the end.

But first:

What is a State Tree?

Wikipedia lists these as the official trees of the US States.

BUT IN THE CONTEXT OF AI, a State Tree is a hybrid of a Behaviour Tree and a State Machine, notionally giving you the best of both worlds. From Behaviour Trees they take the hierarchical structure, from State Machines they take the concept of states and easy transitions.

State Trees let you see at glance what your AI is (or should) be doing at a particular point, and allow you to jump to different behaviours as you wish without having to stick rigidly to the structure of a Behaviour Tree. This makes them very flexible. There’s also fewer concepts involved - no blackboards, no decorators, although there are things that perform similar roles.

This is what a (simple, incomplete) Behaviour Tree looks like:

And this is what (more complicated) State Tree looks like:

The latter is much nicer to reason about, and has most of the information you need available at a glance. I really like them.

What is in a State Tree?

You can skip this part if you just want to get to the headaches.

A State Tree has several top-level concepts:

• Context: an object describing and storing what the State Tree is controlling- typically an actor or an AI Controller.
• Parameters: global variables that you can set and read from, although as of 5.4 setting them at runtime is a bit awkward.
• States: logical groupings of tasks to be executed, and transitions that can happen whilst that state is active. States can have child states (which is where the ‘tree’ bit comes in). States can also have enter conditions that need to be met before entry.
• Tasks: things you actually want the AI to do. These can also provide information to other tasks, or claim resources, and can be run globally- that is, not tied to a particular state. Usually. We’ll get to that. Tasks can also (optionally) finish, reporting either success or failure, by calling their Finish Task function.
• Transitions: these direct execution flow from one state to another. Like states, transitions can have conditions that need to be met before being activated. There’s three main flavours of transition- On Task Succeeded / Failed, which are triggered when tasks succeed and fail respectively; On Task Completed, which is triggered when a task succeeds OR when it fails; and On Event, which has to be triggered manually, either from inside a task or from outside the tree. Transitions can also have a priority, to resolve conflict if two or more are triggered at the same time.
• Evaluators: these provide outside data to the tree. They are essentially pseudo-tasks with no inputs (beyond what is in the context), and that don’t have transitions attached. They can update the information they provide as execution progresses… usually. We’ll get to that in a bit.
• Bindings: using bindings, tasks can expose input variables to be set when their containing state is activated. When you add a task to the tree, you can choose what to bind these exposed properties to from any variables that will be available when that task is executed. You can use the output of an evaluator, a parameter, the context, or another task. This means as the overall state of the game progresses, the behaviour of tasks can be dynamically modified to respond to new information. There is some hidden complexity here however, which we’ll get to later.

So, how do they work? Each tree has a ‘root’ state where execution starts. The State Tree will then try and find states to activate by running the following algorithm, running from top to bottom of the tree.

1. Move consideration to the next child state of the last state checked (which is initially the root state).
2. Check this state for enter conditions. If the conditions pass, or there are no conditions:
   1. This state is marked as active.
   2. If this state is a Leaf State- one that has no child states- stop searching and move to 5.
   3. Otherwise, set consideration to this node, and go back to 1.
3. If the enter conditions fail, move consideration to the next child of the last state checked.
4. If no states remain to be checked, and there are no active states, the tree exits.
5. If the tree is still running, start any tasks on states that are active, stop any tasks on states that are not, and wait for a transition to be triggered.
6. When a transition occurs, move consideration to wherever the transition points to, and go back to 2.

What can possibly go wrong?

Again, I like State Trees. But there are some things you should be aware of so you don’t spend as much time as I did wondering why they’re not fucking working properly.

Headache number 1: multiple active states and tasks

Multiple states can be active at the same time. Multiple tasks can be active at the same time. Each state can have its own transitions.

As mentioned, one of the most common transitions is “On State Completed”, which is fired whenever a task finishes execution by calling the Finish Task function. The problem is that all tasks in active states will receive the notification from all active tasks at or below their own level, meaning that a transition you thought only applied in a parent state may end up firing when something in a child state completes.

In the above example, states A, B, and C will all become active simultaneously- C is the leaf node, but the tasks on A and B will also be started. C completing will trigger the transitions on A, B and C, though because it’s furthest down C will be the one that takes precedence. Probably? Unfortunately, it’s not always obvious, particularly when there’s lots of different tasks and transitions.

This isn’t too much of an issue if you’re aware of it, but it’s not exactly highlighted in the docs, and it brings us to another problem:

Headache number 2: doing two things at once

Say you’ve got two or more tasks that you want to happen at once- for example, you want (A) your character to speak a line of dialogue and (B) walk to a position. Unfortunately you don’t know how long the voice line you’re playing is compared to how long the walk is, so whilst you want both to start at the same time, you want them to complete at different times, and then move execution elsewhere.

You cannot use the On State Completed transition for this. This will fire if either A or B complete, which is almost guaranteed to be premature, causing the task that hasn’t finished to be stopped before completion.

If you’ve got another voice line to play when your character gets to where they’re going and it’s a really short walk, this could lead to the previous line getting cut off, or both playing at once. Likewise, if it’s a short voice line and a long walk, they could start playing their ‘interact’ animation (or whatever) in totally the wrong position.

Again this isn’t too bad if you’re aware of it, and there’s ways of mitigating both of the things in that example outside of the tree, but it’s still something to be worked around. In general there’s two options, depending on how essential that concurrency is.

Option 1: if you know (or don’t really care) that one is going to finish before the other, just make it so one of the tasks never calls the Finish Task function. If only one task is finishing, there’s only one transition trigger. A slightly more complete solution is to subclass the UStateTreeTaskBlueprintBase C++ class and add something like UPROPERTY(BlueprintReadOnly, EditAnywhere) bool ShouldCallFinishTask that lets you set whether it finishes or not in the State Tree overview.

This is what I’ve done in the ExecuteSubtask task I created. I consider ‘sub tasks’ - which is not an official term - to be simple tasks that do something cosmetic or incidental, such as play a sound. They never call the Finish Task function, so they don’t affect the flow of the tree- they just trigger an event on the controlled Pawn, and when execution moves on they simply do any clean up they need to, and then stop.

Option 2: do something more involved, and add a specific task that counts the number of times other tasks below it finish. If you do this you’ll also need to define a transition that isn’t On State Completed to actually break out of this bit of the tree. Event transitions will work for this.

Option 2, however, requires being aware of another possible headache…

Headache number 3: receiving multiple EnterState and ExitState events in tasks residing in parent states

Let’s say you’ve grouped some behaviour together like this:

This is good! This is what State Tree is for! You’ve got a parent State grouping a set of behaviours, and you’ve got a clear sequence of things will happen in the order you’ve specified. If this state gets activated, the tree will immediately start any tasks in A, B and C; when C finishes, execution will go to D, and when D finishes execution will go to E, and then your Evil Space Knight character leaves this particular grouping to go and blow up the Innocent Space Children’s Hospital in the way you’ve defined elsewhere in the tree.

But there’s an issue:

Tasks have three main events that get fired- Enter State, Exit State and Tick. Tick does what you expect, but Enter State and Exit State will be called on any task that is active when a transition happens. In the above example, A, B and C will all become active at the same time, and the associated tasks will get an Enter State at the same time. When C completes, tasks on all three will get an Exit State event.

But when D starts, tasks on A and B will get another Enter State event. When D completes, tasks on A and B will get an Exit State event again, and the same will happen when execution moves to E.

If you’re not doing anything significant in response to those events you’ll be fine, except obviously that’s when want to do a bunch of stuff- setting up and tearing down event listeners, and possibly firing other events off or getting your actor to do something.

Luckily there is a setting for this! In the tasks used in A and B, go to the Class Defaults tab, and uncheck Should State Change on Reselect, which is on by default.

However: this has to be set on the task itself, rather than on the task when you add it to the Tree, so if you want to have tasks that sometimes care and sometimes don’t, you’ll need to extend StateTreeTaskBlueprintBase and add an appropriate flag. Alternatively you can check the TransitionType property on the transition and make sure it’s Changed rather than Sustained, but again this has to be done on the task itself.

Even more infuriating: there’s a case where this just doesn’t work.

Headache number 4: Subtrees, AKA Linked Assets

The previous things on this list are things that are awkward, but mostly a result of the way State Trees work. The next couple are actual bugs.

There’s a great feature in the UE implementation of State Trees that lets you to split off behaviour into sub-trees. To do this, you create a state, and then set its type to Linked Asset. This allows you to specify a different tree, so you can split your logic into more manageable chunks, as well as share groups of behaviours between different AI types.

Big problem: that Should State Change on Reselect property gets totally ignored if you’re running in a subtree, UNLESS you manually recompile the subtree before running it.

Sometimes. Intermittently. It’s annoyingly inconsistent.

So, you quit work for the day, start up the editor the next morning, and find that things aren’t working the way you left them the previous evening. You’ll inspect the tree, make a tweak, recompile… and it’ll work. It’s working again so obviously that was the correct fix. Quit at the end of the day, try again the next day, and… it’ll be broken again. But you’ve worked on something else so it was probably that, right? Right? So you muck around with that for a bit, trying to chase down the problem, until you accidentally recompile the sub tree and OH YEAH THAT FIXED IT.

Except that it didn’t, because it’s not actually anything you’ve done.

This is the worst kind of bug.

I’ve not been able to reproduce a minimal example, but when a tree gets sufficiently complex it seems to pop up. There’s a bug report open for this on the Unreal Engine Issue Tracker.

I figured out a work-around, but you’ll need to get into the C++ to apply it. Create a base class that extends StateTreeTaskBlueprintBase, override both EnterState and ExitState and put in the following implementation (plus anything else you fancy)

EStateTreeRunStatus USTT_EnemyTaskBase::EnterState(FStateTreeExecutionContext& Context,
  const FStateTreeTransitionResult& Transition) {

  if (!bShouldStateChangeOnReselect && Transition.ChangeType == EStateTreeStateChangeType::Sustained) {
    // Unreal has fucked up if this happens, so just exit with what it was going to do anyway
    return EStateTreeRunStatus::Running;
  }

  // Setup any events you want to have on all your tasks etc

  return Super::EnterState(Context, Transition);
	
}

void USTT_EnemyTaskBase::ExitState(FStateTreeExecutionContext& Context, const FStateTreeTransitionResult& Transition) {

  if (!bShouldStateChangeOnReselect && Transition.ChangeType == EStateTreeStateChangeType::Sustained) {
    // Unreal has fucked up if this happens, so just exit without doing anything
    return;
    
  }

  // Tear down any events etc

  Super::ExitState(Context, Transition);
}

Hopefully there’ll be an actual fix for this soon. I’ve tried tracking it down in the source but I’ve not been able to find the root cause, and either way I’ve got a game to write.

Headache number 5: crashes

Another fun thing with Linked Assets: if you put global tasks in them, and that sub-tree has any parameters whatsoever… they crash! Hard! So don’t do that.

You can get around this by moving whatever your global task was to the root node of the subtree, but be aware you may well run into problem described above of getting Enter and Exit events for every single task in the tree. This crash was how I discovered Headache #4. I’ve submitted a bug report. Other crashes have been reported, but this is the one I found.

Headache number 6: Parameters, Bindings, and Tags

Being able to bind things to influence behaviour is great, but bindings may not work entirely how you expect them to. Whilst all bindings are passed as values rather than references, it’s better to think of them as two categories: Value Types - which are floats, structs, Gameplay Tags, and so on - and Pointer Types, which reference UObjects like Actors. Bindings seem to behave like UPROPERTYs, and have a few quirks.

One wrinkle of Unreal is that you cannot have a UPROPERTY that stores a pointer or a reference to a struct- you’ll get a compiler error if you try. Therefore, any structs you have exposed on objects will be passed by value, and copied. If they get updated, you need to make sure the new value makes its way into the State Tree somehow, and that requires a bit of extra engineering.

If you could bind to a function that would be great, but you can’t, so you can’t have something that along the lines of UFUNCTION(BlueprintCallable) float GetLatestValue() to ensure the latest value is available. Blueprint property getters are also ignored if you’ve defined them in C++; they will be bypassed, so existing code may give unexpected results. In general State Tree seems to sometimes behave as part of the Blueprint world and sometimes behave as part of the C++ world, and it can get very frustrating working out which part you’re dealing with at any one time.

You can bind to a property found on an object that a pointer references, but that pointer may wind up being null, and you’ll get a crash. As such you’ll need to make sure that pointer is valid, which leaves us with the original problem.

Evaluators (sometimes) seem to ignore updates driven by Events outside of the ones they come with in TreeStart and Tick. It’s possible some values are being cached, or that the execution context doesn’t always get updated correctly, or it could be a hidden race condition. I’ve not been able to find where in the source code this happens, and again, I’ve not been able to reproduce this reliably. It’s also possible I’ve just done something wrong, but if I have, it’s not remotely clear what.

Properties bound from global tasks always get set… but then you might run into the problems listed in Headaches #4 and #5.

Properties bound from parameters are fine, but as mentioned at the start of this article, updating a parameter isn’t very easy, and is currently impossible in Blueprint. It can be done in C++, but that’s no good for prototyping, and the chances are you’ll be chopping and changing a lot in your State Tree whilst you nail down the behaviour you want.

Of course, if you subscribe to the ‘Waterfall’ model of software development, this won’t be an issue for you, but then you’re also likely living in 1994 and Unreal version 1 won’t be released for another four years.

So, again, a ‘watcher’ task on the root of the tree is the best compromise, assuming you implement the code mentioned in Headache #4 so it doesn’t constantly setup and remove events you need to listen to or produce other unwanted behaviour.

All this is mostly an issue for transitions and state enter conditions- if you want to check against a value to see if you can enter a state or move to another, that value may be stale, and if it’s stale you’ll get the wrong behaviour.

So: Gameplay Tags.

Gameplay Tags are incredibly handy, particularly for replacing enums when choosing states and guarding transitions. However, they are structs, they’re typically passed around in Gameplay Tag Containers (FGameplayTagContainer in C++), which are also structs, and bound structs get passed by value, not by reference or pointer.

All together this means that if you’ve got an evaluator that outputs tags and you’re relying on it for directing the flow of execution, unless you copy them on tick you may wind up getting whatever was there when the tree started.

Not that copying on tick won’t work- it will- but it seems a bit excessive.

If you want to update the tags, you’ll need to do that on the actor you’re controlling- there’s no point updating a copy. Once they’re updated, you need to get them back into the tree and flush out whatever value was there before.

The best solution I’ve found for this is to create an Actor Component that is specifically for holding and updating tags you care about. Give it a GetTags() function and a SetTags(FGameplayTagContainer Tags) function, and an Event Dispatcher that is triggered with new contents whenever SetTags is called. Listen for that event in a global task (or watcher task on the root), use the value in the event to update an exposed variable, and then use that for your transition and enter conditions.

I’ve got a steadily growing library of functions to fill in the missing blanks of the Gameplay Tag implementation- at some point I’ll clean it up and put it on the Marketplace.

Headache number 7: watcher tasks on root

I said this was a solution, right? Well it is, just bear in mind you may need to add a delay task to make sure whatever value you’re driving transitions and enter conditions with is updated before the tree starts using it. This delay can last for 0 seconds, which is the equivalent of the Delay Until Next Tick node in regular Blueprints, but you may find things don’t work without it. Again, when a state is chosen for selection, all the tasks in the selected branch of the tree start at the same time.

---

So, would I recommend using State Trees?

Yes!

Probably.

None of this is intractable, but if you’re happy with Behaviour Trees maybe stick with them for another couple of releases. 5.5 is supposed to be fixing some of this, and they may even be properly reliable come 5.6 or 5.7.

Some quick tips so this doesn’t sound like I’m just bitching about someone else’s code

1. Subclass StateTreeBlueprintBase

There’s a fair bit of repetition when setting up tasks, and usually common things you want some or all of them to do, such as listening for events that let you know when an animation is finished. If you don’t want to constantly be doing that over and over again, and/or you want to be able to have a bit more control over transitions, and you’re not afraid of a bit of C++, subclass the UStateTreeBlueprintBase class. You can also do this in Blueprint, but there’s a lot more that is exposed in C++.

2. You can create an evaluator that returns an Interface.

Sick of casting to your interface? Create an evaluator that does the casting at the start of the tree’s execution and outputs the correct class.

3. You can modify the behaviour of a task by exposing public variables

You don’t need to bind everything to existing variables in the tree. Tasks have three reserved variable categories- Input, Output and Context. Anything in ‘Context’ is bound automatically, assuming the Context is of the correct type. Anything in ‘Input’ needs to be bound, or the tree won’t compile. Anything in Output will be exposed to other tasks.

But, you can simply make a value public- if it’s not in Input or Context, you can then set those values directly by typing them in whilst in the tree overview. You can also still bind to these if you want to.

As an example, we have our Execute Subtask task here with the Subtask Tag set as public:

When I want to use that in the tree, whilst I still need to bind everything under the Input category to an existing variable, I can just type the tag I want to use directly into the task on the tree. One task has now become (potentially) many different ones.

This pattern is great for Gameplay Abilities too… which is what this whole subtask thing should probably be anyway, but that’s for another blog post.

4. There’s a debugger

There’s a debugger. Don’t spam Print Strings when it starts behaving wrong, you don’t need to. However, if you’re using Linked Assets / Subtrees, you’ll need to activate it in the parent Tree.

5. You can use this for stuff other than AI

Want to track a player’s progress through a puzzle? Well, State Trees can do that. I don’t know if you’d want to use this for a full quest system in an open world, but for tracking a particular set of interactions it’s ideal.

6. You can have a dedicated task for deciding what to do next

One place State Trees can start getting hairy is if you have lots of transitions with particular rules in different parts of the tree. You can see this as a strength- after all, being able to see what transitions can happen where just by looking is very useful- but it also makes changing that logic a bit tedious, as you have to remember what you’ve done in different places and make sure you don’t have any conflicting rules.

An alternative is to create a dedicated decision task so you can keep most or all of these decisions in one place.

In this example, consider a watcher task in a parent state that is always running, hooked up so that whenever a Gameplay Tag is changed on the controlled actor’s Gameplay Tag Container it can provide it to the tree, where it can be used in the enter conditions for child states.

The loop then proceeds as follows: pick the State that matches that tag, thus executing the tasks within. When these tasks and their associated states complete, they all have the same transition: go to the decision task.

The decision task gets what the current tag is, checks the actor, player, and world state as required, and chooses a new tag. It then updates the Gameplay Tag Container on the controlled actor - giving the global watcher task a new tag to be used in the enter conditions - and calls Finish Task.

Execution then transitions up to the ‘Melee Loop’ state, and we repeat the process until this NPC manages to blow up the Innocent Space Children’s Hospital, ushering in a new age of Democracy on Cyber Basingstoke Prime.

This isn’t something you’d want to use everywhere, but it does mean you always know where to look when you want to change this particular piece of logic, and you don’t have to pick through the tree and re-bind a load of conditions and transitions.

---

I hope this all helps someone. If it helped you, why not buy my screensavers?. Also, if anyone has any corrections, feel free to get in touch.