Dynamic Attributes and Integer Increments

[shortjared]

Hello,

I'm trying to figure out if it would be possible to model a dynamic map and do integer increments on values in the map. Most my attempts so far have failed with DynamoDB saying the update expression is invalid, since electro puts things in ADD #attr :attr instead of something like SET #attr.#prop = if_not_exists(#attr.#prop, :defaultZero) + :incr

There is the additional issue of deep nesting not being a thing you can do if the item doesn't already exist in dynamodb of course. Ideally it would be nice to have a way to do dynamic top level props.

Playground Link

Part of the problem is needing a dynamic map. This is useful for things like tracking aggregates. Any ideas how I could approach this differently with Electro?

[tywalch]

Hey @shortjared 👋

Are you getting an error when you try to perform an add? I would suspect (given how deeply nested the structure is) that the error is something about an invalid path? The ADD operation (used by ElectroDB) allows for numbers not to exist and will default the values to zero automatically. The challenge is that if the containing map (in your case either summary or testing) does exist then DynamoDB will throw because it doesn't have a map to place the attribute failed. If this is the case, DynamoDB doesn't offer much recourse here, but knowing the error you see would help me understand what's happening there.

That said, if the issue is related to the path not existing (no summary or testing maps prior to your update) here are two ways you might be able to model what you need:

1. Reduce your nested properties by rethinking the interface a tad bit. Perform an idempotent pre-update and then increment properties off a single map

2. Isolate your summary types with entities to simply updates and use collections to simply queries

Let me know about the error message you see, whether what you're seeing is an error path issue, and/or if any of the alternative models I put above work for your use case 👍

[shortjared]

You are correct in the not having the property ahead of time being part of the issue with regards to ADD on the update operation. That's a solvable problem.

Part of the issue is I don't know the names of the properties ahead of time. If Electro had a way to allow "dynamic" patterned top level properties, I could arbitrarily use ADD to track such aggregates. I can't seem to figure out a reasonable way to do it. (I can of course eject to just normal document client, but I lose a lot of the nice bits of using Electro)

core_execution_id: abc
2023-04-28: 198
2023-04-29: 203
2023-04-30: 101

I know this isn't a core concern of the lib, so no worries if there isn't a clean way to figure it out!

[tywalch]

Yup, having explicit top-level attributes is a pretty static constraint currently.

Is your dynamic property a date? If so, have you considering simply making a record for each execution? That would work well sort keys: playground.

If it's truely dynamic (you don't know the value ahead of time) you could also simply add it to your key structure: playground.

I should have suggested this from the start

[tywalch]

You might also try to define maps you need when provisioning the dynamodb table 👍

[shortjared]

Many records in this case is painful because it's a unique need where I may have 10-100 different aggregates on the same record I could update at the same time. It becomes a cost and performance optimization to do everything in a singular write instead of many writes.

Like I said, totally realize I'm trying to get electro to do something it isn't designed for! Thanks for your help in any case, great lib!