Skip to content

Commit

Permalink
mango: rolling execution statistics
Browse files Browse the repository at this point in the history
In case of map-reduce views, the arrival of the `complete` message
is not guaranteed for the view callback (at the shard) when a
`stop` is issued during the aggregation (at the coordinator).  Due
to that, internally collected shard-level statistics may not be
fed back to the coordinator which can cause data loss hence
inaccuracy in the overall execution statistics.

Address this issue by switching to a "rolling" model where
row-level statistics are immediately streamed back to the
coordinator.  Support mixed-version cluster upgrades by activating
this model only if requested through the map-reduce arguments and
the given shard supports that.

Fixes apache#4560
  • Loading branch information
pgj committed Jan 10, 2024
1 parent b275635 commit 2cb7139
Show file tree
Hide file tree
Showing 3 changed files with 270 additions and 37 deletions.
Loading

0 comments on commit 2cb7139

Please sign in to comment.