Once you pass trivial that would definitely pass into the realm of thing I'd want to expose to my tests. "Starting from this state and given this set of inputs do I get to this state?" is a pretty basic test, and maybe there's a bunch of people way smarter than me who can naturally run complex state machines in their head, but I find myself frequently surprised by at least one thing they do and I do not generally just splat them down into the code correctly on the first try.

In the cases where I do this, it's when I try really hard not to actually expose the state machine to tests anyway because there's a better way to hit them in tests with their expected inputs and outputs rather than an implementation detail of the state machine itself.