-
Notifications
You must be signed in to change notification settings - Fork 344
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tolerate runtime configuration instances named 'local' #2968
Conversation
Thanks for making a pull request to Elyra! To try out this branch on binder, follow this link: |
@kevin-bates could you add to the description how the info will be submitted (or expected) when local and non-local? (e.g. runtime-config and runtime-schema) |
From the description...
The schema for the runtime (and thus which processor to use) comes from the runtime configuration's instance referenced by |
@kevin-bates I believe this is where you would set the |
Thanks @karlaspuldaro - that's the very location I had identified as well, so it's great reassurance! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code looks good to me :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM based on spot testing
Please give me till tomorrow to review it, thanks. |
All the confusion described in #2964 seems to be related to the UI only propagating the runtime_config which is the key to finding the runtime configuration profile, and when an existing runtime tries to use it as local it becomes problematic. How about, instead of using workarounds, we don't actually also propagate the runtime type, which is how the ui finds all the runtime configuration available for each runtime type thus we just need to make sure the backend uses both runtime type and runtime config to find the proper processor to execute the pipeline? NOT A CONTRIBUTION |
Thanks for your review @lresende. Adding The "workaround" is due to the fact that we use a "magic config name" named 'local' to indicate that the pipeline is run locally. The source code was already using a "None" or "local" approach when checking |
When submitting pipelines to Kubeflow or Airflow runtimes, Elyra runs the pipeline locally if the corresponding runtime configuration is named "local" as described in #2964. This is because the server looks at the runtime_config field in the pipeline for either no value (
None
) or the name"local"
. This pull request implements changes such that only invocations for local processing will submit pipelines with no value forruntime_config
.Should we decide to promote the LOCAL runtime to a full-fledged runtime (where there's an associated schema and support for runtime configuration instances), most of these changes should still work since there will be a value for
runtime_config
in these cases, which will then determine the target pipeline processor from the referenced schema name within the runtime configuration instance. We should only then need to make changes where we set theruntime_config
toNone
during submission (and in both the UI and CLI applications).Still to do:
runtime_config
tonull
(orNone
) in the UI when submitting pipelines for local executionResolves: #2964
Developer's Certificate of Origin 1.1