Storylet API & macros

[tmedwards]

@ChapelR and I have been working on a native storylet API & macro set for SugarCube.

What's everyone's feeling on the following proposals?

---

Storylet API Specification

Storylet Static Members & Methods

Storylet.add(passageName | []passageNames, definition)

Register storylet(s). Reusing a name causes the original storylet definition to be replaced.

Definition object:

A storylet definition object should have some of the following properties:

• conds: (optional, [](string | function)) An array of the storylet's own conditions.
• groups: (optional, []string) An array of named condition groups that should also be checked when determining availability.
• priority: (optional, integer) The priority given to the storylet when Storylet.available[Storylets]() is called. Priority is the relative importance of the storylet. When checking availability, only storylets with a priority equal to the highest priority seen during that check are yielded. Defaults to 0.
• weight: (optional, integer) The weight given to the storylet when Storylet.random[Storylet]() is called. Weight is the relative weight of the storylet when calling for a random storylet. A higher weight increases the chance of the storylet to be chosen. Defaults to 1.

NOTE: Conditions have access to the current storylet passage name via the _storyletPassage temporary variable. Additionally, function conditions are passed the current storylet passage name as their first parameter.

NOTE: Specifying no conditions is equivalent to a single condition of true—i.e., the storylet will always be available.

Storylet.all() → []string

Returns an array of passage names for all registered storylets.

Storylet.allStorylets() → []<Storylet>

Returns an array of all registered storylets.

Storylet.available() → []string

Returns an array of passage names for all storylets whose conditions yield true.

Storylet.availableStorylets() → []<Storylet>

Returns an array of all storylets whose conditions yield true.

Storylet.delete(passageName) → boolean

Deletes the storylet with the given passage name and returns whether the operation was successful.

Storylet.get(passageName) → <Storylet>

Returns the storylet with the given passage name, or null on failure.

Storylet.has(passageName) → boolean

Returns whether the storylet with the given passage name exists.

Storylet.isAvailable() → boolean

Returns whether any storylets are currently available.

Storylet.random() → string

Returns a randomly selected passage name from the storylets whose conditions yield true or an empty string, if no storylets are available.

Storylet.randomStorylet() → <Storylet>

Returns a randomly selected storylet from the storylets whose conditions yield true or null, if no storylets are available.

Storylet.size → integer

Returns the number of existing storylets.

Storylet.conds Static Members & Methods

Storylet.conds.add(groupName, []conds)

Register a group of conditions to be refenced by storylets via their groups properties. Reusing a name causes the original group to be replaced.

NOTE: The special name :all may be used to register a group that applies to all storylets without needing to be refenced via their groups properties.

Storylet.conds.delete(groupName) → boolean

Deletes the condition group with the given name and returns whether the operation was successful.

Storylet.conds.get(groupName) → []conds

Returns the condition group with the given name, or null on failure.

Storylet.conds.has(groupName) → boolean

Returns whether the condition group with the given name exists.

Storylet.conds.size → integer

Returns the number of existing condition groups.

<Storylet> Instance Members & Methods

<Storylet>.conds → [](string | function)

Returns an array of the storylet's own conditions.

<Storylet>.groups → []string

Returns an array of the named condition groups associated with the storylet.

<Storylet>.passage → string

Returns the storylet's passage name.

<Storylet>.priority ↔ integer

Provides access to the storylet's priority.

<Storylet>.weight ↔ integer

Provides access to the storylet's weight.

<Storylet>.isAvailable() → boolean

Returns whether the storylet's conditions yield true.

---

Storylet Macro Specification

SEE: The Storylet API Specification (above) for specific details on some of the following features.

Registering storylet passages.

<<storylet passageList>>
    [<<cond conditionalExpression>>]
    [<<func function>>]
    [<<groups condGroupNames>>]
    [<<unique>>]
    [<<priority integer>>]
    [<<weight integer>>]
<</storylet>>

NOTE: Conditions have access to the current storylet passage name via the _storyletPassage temporary variable. Additionally, function conditions are passed the current storylet passage name as their first parameter.

Defining storylet condition groups.

The special group name :all automatically applies to all storylets.

<<condgroup groupName>>
    [<<cond conditionalExpression>>]
    [<<func function>>]
    [<<unique>>]
<</condgroup>>

NOTE: Conditions have access to the current storylet passage name via the _storyletPassage temporary variable. Additionally, function conditions are passed the current storylet passage name as their first parameter.

Using storylets.

Link to a random available storylet. If called multiple times within a turn, will not yield duplicates.

<<storyletlink [linkText]>>

Include a random available storylet.

<<storyletinclude [elementName]>>

---

EDIT(1): Made a few clarifying edits to the OP. I'll make some more as I have time.

[brwarner]

I really like the idea of "condition groups". When messing around with the new Harlowe implementation I found myself retyping when $location = 'The Atrium' and the like a lot and I could see a lot of errors creeping in from typos in oft-typed expressions.

The Macros in your proposal are a little under-documented. For users that are less Javascript savvy it might be easier for them to provide feedback if they could get a robust understanding of how everything would work just from the description of the Macros.

Speaking of the macros, from what I see here it seems that I'd have to have some kind of master passage run at startup that has one <<storylet>> macro per-storylet to register them all? I see it takes a list of passages as its argument, so I suppose I could register many at once but only if they all share the same properties. I could see this getting a little cumbersome. Harlowe's implementation is pretty slick but it is built atop their new (metadata:) system which I don't think Sugarcube has any equivalent of. I understand why you might not want to go that way because there's no infrastructure for it yet but it might be something to seriously consider. I could see metadata being useful for other features, especially since tags are so limited in Twine right now.

It's also a little unclear to me what the difference between priority and weight are? Does priority work like the exclusivity value in Harlowe?

[brwarner]

It's a copy of the base specification we're working from, not documentation. Though your point is taken.

Right. That's what I should have said.

SugarCube doesn't really need an equivalent of that because SC doesn't restrict authors from doing the same thing—in whatever flavor they want; meaning in data type and/or statefulness.

Agreed. Your Javascript API gives authors everything they need to add their own automated registration based on whatever criteria they like (searching through tags, keywords in the title, scanning some hidden content of the passage text, etc.). I still think it would be nice if in addition there was some built-in way to flag a passage as a storylet and embed the conditions/groups/priority/etc. within. I admit that may be beyond the scope of this change, though, as it'd require making a macro that is somehow executed wherever passage it may be in on startup but has no effect when the passage actually runs.

Priority is...

Thanks for clarifying! That's what I was hoping they meant. And thank you for updating the specification.

Excited to see this implemented.

[tmedwards]

Addendum. As an example of priority. Let's say that four storylets passed their condition tests: two with the default priority of 0 (A, B) and two more with a priority of 5 (C, D). Only those whose priorities equal the highest priority value seen (5), of the passing storylets, will be yielded (C, D).

[tmedwards]

I still think it would be nice if in addition there was some built-in way to flag a passage as a storylet and embed the conditions/groups/priority/etc. within. I admit that may be beyond the scope of this change, though, as it'd require making a macro that is somehow executed wherever passage it may be in on startup but has no effect when the passage actually runs.

Processing every normal passage looking for storylet definitions comes with a few cons:

1. The more passages you have, the worse your startup time gets. No, we're not talking about epic times here, but IME people really don't like delays in their web games/stories.
2. It makes having sets of passages with the same conditions a pain, since you have to duplicate the macro invocation within each passage.
3. (Twine 2 specific) Make a bobble in the conditions of many passages and you're in edit hell. You could use its find/replace, but that thing is, unfortunately, quite literally dangerous.

The only real pro putting storylet definitions within their passages has is that it keeps the definition close to the content. As noted above, however, that's not always a net plus even if you like that arrangement.

[brwarner]

Those are all great points. I haven't used a storylet system on a large scale project as of yet so maybe I'm not thinking enough about logistics and scale. Thanks for the reply.

[ChapelR]

Just to chime in on this, I think the limitations of defining the storylet conditions in the passage, particularly since games with heavy storylet usage will likely have many with overlapping conditions, make it less useful for larger-scale games. And having two ways to define storylets is in my opinion a bigger potential usability issue than the one it would purportedly solve. I also think not having to pre-process all the passage content looking for storylets makes more sense generally.

Separating content from conditions also has benefits in its own merit, as briefly touched on by TME. It creates a separation between content and the underlying logic that, while Twine doesn't generally enforce, is probably wise, maybe even necessary, for larger-scale games.

That's not to say Harlowe's implementation (or other QBN implementations based on passage content or metadata) should adopt this model, just that I think the benefits outweigh the drawbacks, and that whether the drawbacks are drawbacks at all is a matter of preference anyway.

Still its good feedback, so we appreciate it. I think this aspect of the storylet system is unlikely to change though.

[brwarner]

A thought on naming:

Is layer a better name for what you want to capture with priority? Priority, to me, doesn't immediately imply what you want to capture here (Harlowe's "exclusivity" is also not intuitive). I find it easy to confuse with sorting order, since that's also in play. I like layer because occlusion is baked into the concept. If I bake a cake with layers, I expect to see only the top layer, and if multiple elements are on the same layer, I expect them all to be visible.

[ChapelR]

I'll be honest, "layer" doesn't make a lot of sense to me. It is a way to visualize it, and it might be a useful metaphor for what's happening, but "priority" is the correct term I think. If priority can be a little inaccurate, maybe "precedence" or "importance" instead. Though like I said I think priority works fine. If you didn't know what it did at first upon reading it, I'm curious as to what you're assumption was about what it meant.

[brwarner]

I think its because you have both priority and weight and while it's obvious weight doesn't mean what priority signifies, I find it easier to think priority might mean what your weight signifies. If I was given a set with "priorities" and told to pick one at random, my first instinct would be to assume priority meant something like likelihood. Obviously since you also have a weight for each item, it has to mean something else. For whatever reason, the idea of priority in random selection doesn't make me think those items have to come out first. As I type this, I realize that maybe this is just a me thing. Perhaps everyone else immediately associates priority with what it's meant to mean here.

[skyvert]

I just stumbled across this tonight. I have also been working on a system like this, and I had no idea that other people were working on similar systems or that there was a vocabulary built up around this concept. A friend of mine says "There are no new ideas!" But I disagree, I think new ideas happen constantly, but that many people get the same new idea at almost the same time.

I have to say, I'm very, very happy to see that my ideas for story generation (carving down a set of possible paths, then choosing one) were not insane, and that many other people were thinking along the same lines. I am so, so relieved that it will not be impossible to communicate the idea to others.

The system I have been working on is very similar to this one here. The core philosophy is the same, but it differs in a few small, significant ways. I will briefly detail how my system differs from yours, and I will briefly give some small critiques of the system proposed here. Feel free to ignore my suggestions if you disagree.

Differences:

• my system uses tags on Passages to define preconditions.
• instead of functions or groups, it has 'named preconditions' which are special passages which can be passed args if desired.
• my system can run lambdas after a selected Storylet displays. I don't think this is an important feature and I may remove it.
• my system does not pass the current passage name to the preconditions. I might add that feature, it feels maybe very useful.
• my system does not have weighted-random
• my link and include macros take another arg, either a passage name prefix, or a tag name, as the initial filter on the list of all passages.
• my link and include macros take an arg saying how many candidate passages to link or include, default = 1
• my link macro will, instead of linking, include any passage tagged 'include' (this lets passages provide their own link text)

Overall, the major philosophical difference is that I very much aim to keep the definition close to the content. I want this because the main goal I had was de-coupling story passages from each other, to encourage game modding by a dis-organized team of contributors.

Critiques:

• definition is not close to content. large projects and team projects will benefit from maintaining locality.
• the groups and functions features overlap. only one is necessary.
• the set of Storylets that you have to filter can be greatly reduced by passing a passage name prefix or tag name.
• the link macro is not able to let Storylets provide their own link text.

[tmedwards]

Cc: @ChapelR . In case he's got any feedback.

I just stumbled across this tonight. I have also been working on a system like this, and I had no idea that other people were working on similar systems or that there was a vocabulary built up around this concept.
Frankly, the whole QBN (Quality Based Narrative) nomenclature is pretentious codswallop for what is essentially an event system, but that's what an influential article on the subject related to IF used, so….

You may want to look at Tiny QBN for Twine/Sugarcube for another take on such a system—which sounds more like your system on first blush.

The system I have been working on is very similar to this one here. The core philosophy is the same, but it differs in a few small, significant ways. I will briefly detail how my system differs from yours, and I will briefly give some small critiques of the system proposed here. Feel free to ignore my suggestions if you disagree.

Differences:

• my system uses tags on Passages to define preconditions.

So does Tiny QBN, so you're not alone—it also uses comments (described below). I feel it's silly. I don't know how you're handing it, but, based on Tiny QBN, it pushes far more detail/work into tags than they're meant for, which leads to all sort of nasty conventions and restrictions to make it work.

• my link and include macros take another arg, either a passage name prefix, or a tag name, as the initial filter on the list of all passages.

I've been considering adding a use-time filter conditions—pre- and post-.

• my link and include macros take an arg saying how many candidate passages to link or include, default = 1

I prefer to have the macros do one thing. They can simply be called multiple times if desired.

• my link macro will, instead of linking, include any passage tagged 'include' (this lets passages provide their own link text)

The same may be achived via <<link>> + <<replace>> or <<linkreplace>> containing a <<storyletinclude>>.

Overall, the major philosophical difference is that I very much aim to keep the definition close to the content. I want this because the main goal I had was de-coupling story passages from each other, to encourage game modding by a dis-organized team of contributors.

Critiques:

• definition is not close to content. large projects and team projects will benefit from maintaining locality.

Keeping storylet passages and their conditions together as a single unit can be advantageous, certainly—especially for your noted use case. That said, it also comes with some downsides:

• If you allow unfettered conditional logic—tags are not going to handle that—then you're either stuck with a few options, none of which are particularly good:
  • Using a macro within the storylet passage that wraps both the conditions setup and content, which runs the appropriate one when necessary—i.e., the former at startup and the latter during play.
  • Parsing a comment within the storylet passage for your logic.
• Assigning common sets of conditions to storylets becomes somewhat onerous. Requiring you to either repeat yourself or define named preconditions for every common set of conditions, assuming it works as I'm guessing—admittedly, the latter probably isn't horrible, and you can do the same with this system, but I don't think authors should be required to either.

This critique is also Twine-centric, which I find odd considering your focus on large and/or team projects—neither of which are well served by Twine. Users of Twee compilers, which make collaborating far more easy in the first place, would find it simple to place the storylet definition near its content passage.

• the groups and functions features overlap. only one is necessary.

You seem to have misunderstood something. Based on its name alone groups are analogous to your "named preconditions". Functions are simply another way to write conditions, and while there is overlap with conditional expressions, they're distinct enough—and supporting both isn't onerous.

• the set of Storylets that you have to filter can be greatly reduced by passing a passage name prefix or tag name.

Conditions are filters. You're simply adding another to the set, which means you're still filtering the entire set of storylets at some point. Well, unless you're binning them during definition—which is definitely an idea, but seems like make-work.

Anyway. As a general criteria, storylet conditions can handle that at the storylet level. If it's needed at use-time, then as I noted above, I'm considering adding a way to do that.

• the link macro is not able to let Storylets provide their own link text.

That's true. It's something that I could add.

[ChapelR]

Even in the Twine GUI, it's easy enough to group passages together. You could even link them if you wanted. I think the mild inconvenience of external passages defining storylets is a pretty small price to pay for the improved flexibility and other more engine-side benefits. I know tags are used by TinyQBN as mentioned by TME but I don't think tags are a good place for this. If I were persuaded to want to keep the content and conditions together, I'd probably use a macro set much like the one we have now, and maybe a single tag to define the passage as a storylet in the first place.

I also think the opposite of your assessment is true. As a general rule, separating content from the functionality and logic makes larger games simpler. I believe this is the case in this system too. Assume you had a set of storylets that all shared a certain condition, and you made a logic error in that condition. With the logic separated out, you go to one passage and make the correction once. With the content and logic shared, you have to go to each affected passage and make the fix on all of them. This has several more fail points.

Likewise, being able to combine conditions quite easily and define groups or sets of storylets all at once, both things larger and more complex games would want to do, is vastly simplified in what we've got here. I actually think smaller-scale games with lower numbers of storylets are probably being inconvenienced more.

And this system can and will coexist with already existing storylet/QBN systems, most of which function by including the logic in the content passages. I don't feel this is a scalable or convenient approach for everyone, but it is a totally reasonable approach that users may prefer.

[Gamewise]

Hey this looks super cool and useful!

Happy to do some video tutorials to help ppl use these features when this comes out (if that'd be helpful etc) 😊👍

[Akjosch]

I'd appreciate it if both priority and weight were (optionally) functions, with a StoryletContext (includes the current game state, essentially, but also which storylet it's being called for) as a parameter. In a similar way with what I'm doing with my random event engine example, this allows for some storylets to become more or less "important" based on the game state.

[tmedwards]

As to letting them also be expressions/functions—similar to the conditions themselves. That seems like a perfectly reasonable request.

As to the context object. Since they would be executed within SugarCube's user scope they'd already have access to the game state and APIs, so a separate context object duplicating that doesn't seem necessary.

[Akjosch]

I also don't think a context object encapsulating the state is strictly necessary, I simply prefer to use pure functions whenever possible. Either way will of course work.