Object ID System

[pcen]

There has been some discussion regarding object IDs and GUIDs which are currently used to identify cities, map units, and players in this CFC thread. This PR also has some relevant discussion.

Let's weigh the pros and cons of some options such as GUID, incrementing ID, etc. and adopt a standard to implement moving forward.

[QuintillusCFC]

Tagging @JimOfLeisure , @WildWeazel , @maxpetul as having contributed thoughts on this subject in the past.

One advantage of GUIDs is they are globally unique - in multiplayer, each computer could generate a GUID for new player units, and it would be very unlikely that they could collide. The disadvantage is GUIDs are probably the worst option for human readability.

Human readability influences my next point - if we want the save/mod format to be human modifiable (without special tools like scenario editors), the things that humans might want to modify should be in a format they can read.

To that effect, for terrain types, we started using keys. These are not going to be generated in-game, so it isn't a concern there, but a human might want to modify them in the JSON save/scenario file. The keys are human-readable such as "plains", "ocean", and "grassland". You could add new ones such as "taiga", "savannah", etc. (although right now C7 wouldn't know how to use those new ones).

For tiles, an obvious key would be a combination of its X and Y coordinates. If a human wants to add a unit to a tile, being able to refer to it by coordinates would be much more convenient than by a GUID.

But for other things, such as units or buildings-built-in-cities, there may not be an obvious "key". I think for these, an integer ID or perhaps even a GUID may make more sense.

So my vote may be that it depends on the object, whether it will be generated in-game, and how likely it is that a human will be visually cross-referencing references to it.

[pcen]

An alternative to GUIDs that would address the issue of length and relatively poor comparison performance would be something like twitter snowflake IDs, which (should) ensure that different clients won't generate the same ID.

If we stick with GUIDs, we should be using the Guid struct instead of storing them as strings for performance.

[QuintillusCFC]

I hadn't heard of snowflake IDs before (Wiki link: https://en.wikipedia.org/wiki/Snowflake_ID), but it might be appropriate. The machine ID part would be useful in multiplayer, and the millisecond-based time component and sequence part should be more than adequate. It looks like they are stored as just a long string of digits, too? So something that could be stored in JSON.

I'm curious about the performance difference? How significant is it? I could see 64-bit snowflakes being quicker than 128-bit GUIDs since it would fit in a regular 64-bit int, but I'd be somewhat surprised if GUIDs were noticeably slow.

Good to know there's a GUID object in C#.

[pcen]

It's probably not much faster than a Guid struct. The main problem with using Guid.ToString is that strings are a reference type in C#, so comparing them is a deference + a string comparison in contrast to Guid structs which are a value type

[JimOfLeisure]

In my early coding, I just punted and used IEquatable<T> as a future-proof identifier and used ints in the interim.

I still don't have an image in my head of how GUIDs would be used. I know we don't want limits, but even a signed int32 gives us over 2 billion positive identifiers. I know we don't want limits, but 2 billion instances of each class type?

Would a default 'plains' tile description have a GUID? Would that be different for each mod? Each game? Does each unit in a game really need a GUID? Seems like a player ID and auto-incrementing int would more than suffice. I could see mods having GUIDs, but within a mod or within a game I'm just not seeing that the universe extends beyond either, so 'global' is very limited in my estimation. If you wanted the ability to combine game save worlds into a solar system or galaxy...maybe GUIDs make sense and units can travel between worlds. But short of that? Meh.

[WildWeazel]

Apologies for letting this sit for so long, but it's time we figure it out.

@maxpetul I don't necessarily disagree with you, but could you elaborate on your objections to GUIDs from the CFC thread? Firstly by GUID do you mean the System.Guid struct specifically, or 32-hex strings, or any long random alphanumeric ID? And is the problem with multiplayer sync that each player would have to produce the same GUIDs? Because you can (re)construct them from known values, and we could make the host responsible for initializing all data.

I was also unfamiliar with snowflake IDs as a formal thing, though I've seen similar schemes before. That seems to address the performance and sync concerns, though it's not much more user-friendly for defining content. As I'm thinking more about it though, as @QuintillusCFC touched on above I think there's a case for 2 different types of ID:

1. A user-defined and (if desired) semantic ID for static content in a mod/ruleset, such as a civilization or a unit class or a technology. These are singular objects, or at most prototypes, that only need exist once per game and should ideally be globally unique across all C7 content so that they can be combined in mods while leaving room for overloads or translations of the displayed name. This could be any alphanumeric string, for which we could recommend a naming convention: something like "MyAwesomeC7Mod.Unit.UnicornRider". The benefit of being human-readable is that this is the ID that would be used in mod files to associate things like dependencies and effects. Since you're not going to be searching through large collections of these all the time, string comparison is less of an issue. If there's a name collision, that's a user/mod error that can be called out and rejected on load.
2. Something like snowflake IDs or a hash as mentioned already, for things that are generated in-game and need to be unique to a single game instance: units, cities, tiles (a bit of an exception since they have unique coords), etc. This would be more performant and practically guaranteed unique, for large data sets that need to be searched. Even an integer would seem to suffice here since as @JimOfLeisure mentioned we're not dealing with vast numbers of anything, though I could see the benefit of a hash.

[maxpetul]

My biggest objection to GUIDs is that they need to be synchronized across clients in multiplayer, so I'm not too worried about structs vs strings (though I'd choose structs). We could surely come up with some scheme to synchronize them but I'd rather choose a design that doesn't have that problem at all, like serial numbers. I think the synchronization problem reflects the fact that GUIDs are just the wrong tool for the job. We don't actually want IDs for in-game objects to be globally unique, instead we want them to be shared by all clients in multiplayer. I also don't like how long and ugly GUIDs are. For example they visually clutter up our logs. Another example, if we were to display them in the UI, say in an error message, that text had better be made selectable somehow b/c no one's going to retype a GUID. Long IDs could be justifiable if they at least carried some useful information, which GUIDs don't. I don't have a problem with assigning GUIDs to out-of-game things like entire mods, but for in-game objects like units, I think serial numbers are in every way a better choice.

[WildWeazel]

I'm not following how synchronization is a problem with GUIDs but not with serial numbers.

[maxpetul]

If all clients spawn a unit and each generates their own GUID, they'll all end up with different GUIDs, by design of course. Whereas if they all tick up a serial number, they'll all end up with the same serial number, assuming their game states were all identical, which they had better be.

[QuintillusCFC]

It sounds like the question might boil down to, is each client running the simulation or is there a client/server model?

If each client is running the simulation, then I agree with maxpetul, GUIDs wouldn't work for that.

If it's a client server model and only the server is running the simulation, then I don't see any problem with GUIDs for synchronization, and thus agree with WildWeazel.

I'd been kind of thinking we'd favor one player's game running as the "server", but that's largely because that's how I've seen it done in most games where I'm aware of how it's done. Also it seems like it might simplify the arbitration of whose action is done first, in cases where that matters (e.g. simultaneous turn MP).

But even if we do go with the server model, I agree with maxpetul that GUIDs are not human-friendly. I'd much rather see in the logs "Warrior with ID Carthage-12 attacks" than "Warrior with ID 218b3-8a389-be281-52ef1 attacks", especially since GUIDs are usually even longer than that example. There's nothing about Carthage-12 (or a similar human-readable scheme) that I don't think would work that GUIDs would cause to work.

Since part of the goal is that it should be easy to mod without specialized tools, I think having something that's easy for humans to type and remember is useful.

[QuintillusCFC]

I largely concur with what WildWeazel wrote. Though for the static content one I'll note that it probably isn't necessary or feasible for unit prototypes/technologies to be globally unique, just unique within a ruleset. E.g. You could have two different mods/scenarios with a unit prototype called "galleon" and the stats could be completely different.

Hypothetically, if we somehow allow combining multiple sources of unit prototypes into one rule set, and both define a "galleon" unit, that could cause the combination to be rejected on load, but I'm not sure when we'd be drawing that together? I guess the question is does the benefit of "MyAwesomeC7Mod.Unit.Galleon" outweigh the extra overhead versus just plain "galleon"?

[QuintillusCFC]

In terms of scenarios, I'm favoring just plain "galleon". It's if we think about a fully-integrated Steam Workshop style infrastructure that I think it could have some value. Maybe WildWeazel and Quintillus both create units called Galleon, then you could create a mod and refer to WildWeazel.Unit.Galleon, and it could automatically download the correct one on install.

The problem this would potentially solve is that in Civ3, unit file names are tied to the in-game name, so if you want to call it something different, you have to rename the .ini file and the structure in general.

Overall though, I'm still not convinced it's worth the complexity over plain-old "galleon". In my editor, I did another work-around for this with the "CFC Workshop" where you can download units in-editor, and it renames the files to what you want them to be called, and you're ready to go. It doesn't solve the I18N aspect of things, if you want a scenario to support multiple languages, but we can allow that via JSON and still have the unit prototype ID be "galleon".

---

Part of the reason I don't think it's worth the extra complexity is the "we already have enough on our plates" problem. But I also realize now that there's a distinction between the unit prototype in the mod, and the unit art assets, such as would be added to the Downloads Database. I think there is a better case to be made for a longer, unique ID for the art assets themselves, as that could solve the problem of having 17 copies of, say, a Comanche Horse Archer because you have 17 mods that use that unit. The "horseArcher" prototype in the mod could have a property, "artAssets", that points to "deltastrife.units.comancheHorseArcher".