Event Notification Extension

[rhc54]

Signed-off-by: Ralph Castain rhc@open-mpi.org

[rhc54]

@dsolt @jjhursey @abouteiller Please comment - I'd like some feedback before doing the prototype implementation

[jjhursey]

I think this sounds ok.

One of the points made was the need for events to be sent between threads within a process. That seems difficult to manage since we would have to track which threadid registered the request, and enforce that only that threadid receive the notification. And multiple threads could be registered for the same event so that would need to be tracked.

That seems like a lot of bookkeeping to me, but maybe I'm thinking about it wrong.

[rhc54]

I agree about tracking at a thread level - probably outside where we want to go, and I should clarify the comment. My intent was only to support multiple registrations against the same code or group of codes. It would be up to the user to ensure that each thread registered a different callback function.

This was the mechanism, when combined with a new data range of "proc_local", I had envisioned for someone to "notify" a separate thread of an event such as declaration of a programming model.

[abouteiller]

Shouldn't the default behavior be to prepend? In a typical use case, the application will initialize libraries in a "more generic to more specific" order, meaning that the last registered handler is the one that is the most "specialized". So it most often should have precedence at trying to "fix the problem", before passing the hand to more generic handlers.

I understand that this additional PR greatly alleviates this concern, but it still seems more logical in that direction to me.

[rhc54]

I was reflecting the current default behavior, but I am open to modifying that if desired. Let's discuss at the meeting and see what people think.

[abouteiller]

I do not remember if registering twice an handler is already illegal or not. If it is not, it certainly should become now, as otherwise the before/after relation is ill defined.

[rhc54]

The before/after relation is only defined for the specific registration request that contains it. So if you register another handler for the same code, then it must include its own relational directives. I will clarify, however, that you cannot register more than one handler with the same string identifier.

[abouteiller]

I think the logic is reverse here:
lets consider a chain A B C D(default)

We call A(status, attr={})
if A can handle the issue reported by status it changes the attribute to NO_TERMINATION.

We then call B(status, attr={NO_TERMINATION})
B cannot handle the condition (it indicates a fatal error of some sort that cannot be corrected), with these definitions, B cannot reset to terminate.

I propose the opposite logic:

1. A(status, attr={DEFAULT_TERMINATION})
   A is happy, so it sets NO_TERMINATION
2. B(status, attr={NO_TERMINATION})
   B is unhappy, so it sets TERMINATION
3. C(status, attr={WANT_TERMINATION})
   C is happy or unhappy, doesn't matter, WANT_TERMINATION cannot be overridden
4. D is default, sees WANT_TERMINATION, terminates.

Now, if all handlers are happy, we'd reach
D(status, NO_TERMINATION)

[rhc54]

So basically what you are proposing is that we allow each handler to modify the collected array of results, as opposed to only adding to them? We would definitely have to define which attributes can be overwritten, but implementation would be simple and not require an API change, if that's what people want to do. Again, worth bringing up in the meeting.

[rhc54]

@abouteiller @jjhursey I have updated the RFC to reflect your comments - please review

[jjhursey]

This looks good. I still think we need to be careful about wording around delivering an event to a specific thread. That requires tracking of thread ids and making sure we deliver the event in the context of the same thread. I'd like to make that the caller's problem to solve, not PMIx. I think the wording in here is appropriately flexible.