Design of Invokers

[JehandadKhan]

The internals of our library went some major changes recently with solver registry, invokers etc. It would be helpful to have a design document to understand what the new design looks like.

Motivation and basic concepts

See Extend Solver/Solution architecture with Invoker objects. #1127

Design of Invokers

Initial design is explained in the description of PR #2192, Invokers core and Forward Direct

Further reading (more code than text 👀)

• Evolution of low-level design in detail can be found in the same PR, beginning from this comment and below. There are lots of informative comments that touch different aspects like:
  • Why we want to hold Invokers separately from the kernels
  • Measuring kernel execution time
  • Invokers and moving towards Find API 2.0
  • A notion of Invoker Factory
  • ...
• Core Bwd Invokers + implementation for Backward/ Direct algo can be found at PR #2366.
• Core of WrW Invokers + implementation for WrW/ Direct algo: see PR #2454
• Why and how computations should be moved from Invoker to InvokerFactory: see discussion in PR #441.
• Capturing values in Invoker/Factory labdas: be careful when using slice and similar things, see note in PR #800.

Please do not hesitate to ask specific questions.

[atamazov]

On FP16_ALT support in Invokers

FP16_ALT requires support for selection of "FP16 mode" during kernel invocation. The necessary parameter is passed to the Invoker and it is the Invoker who must properly handle it. How it should be done is a design choice:

• One way, the programmer adds ALT precision mode to some existing kernel and pass the "ALT" parameter to it.

• Another method is creating a new kernel that implements the ALT mode and run it instead of the "normal" kernel. Invoker looks at the "ALT" parameter and runs appropriate kernel. What is important here is that both "normal" and "ALT" kernels must be always included into a Solution. This way, both "normal" and "ALT" kernels are built and then ready to be used in the Invoker.

[atamazov]

On usage of member functions in Invokers

Only static member functions are allowed to use from within the Invoker (or you must instantiate the class first). The simplest is to use the normal function instead. More info at #1971.

[atamazov]

[Attribution] @junliume @johnny-keker Please set https://github.com/ROCmSoftwarePlatform/MIOpen/labels/specification and remove https://github.com/ROCmSoftwarePlatform/MIOpen/labels/request_for_comments

Please also assign @iq136boy @illsilin @JehandadKhan

[atamazov]

Pitfall with capturing slices

Capturing values in Invoker/Factory labdas: be careful when using slice and similar things, see #800 (review)