Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RunnerScriptConstructor refers to wrong inputs.parameters as an inline template #630

Closed
2 of 4 tasks
elliotgunton opened this issue May 17, 2023 · 0 comments
Closed
2 of 4 tasks
Labels
type:bug A general bug

Comments

@elliotgunton
Copy link
Collaborator

elliotgunton commented May 17, 2023

Pre-bug-report checklist

1. This bug can be reproduced using YAML

2. This bug occurs when...

  • running Hera code without submitting to Argo (e.g. when exporting to YAML)
  • running Hera code and submitting to Argo

Bug report

Describe the bug
A clear and concise description of what the bug is:
Use a "runner script" constructor, and make the script an inline template using the workaround in #628:

@script(constructor="runner")
def print_message(message: str):
    print(message)

with Workflow(
    name="hera-inline-test",
    entrypoint="d",
) as w:
    with DAG(name="d"):
        task = print_message(arguments={"message": "test"})
        task.inline = w.templates.pop()
        task.template = None

You can then see in the templates generated yaml:

  templates:
  - dag:
      tasks:
      - arguments:
          parameters:
          - name: message
            value: test
        inline:
          inputs:
            parameters:
            - name: message
          name: print-message
          script:
            args:
            - -m
            - hera.workflows.runner
            - -e
            - herainlinetest.workflow_template:print_message
            command:
            - python
            image: # <private image>
            source: '{{inputs.parameters}}'
        name: print-message
    name: d

The line source: '{{inputs.parameters}}' is actually referring to the dag's {{inputs.parameters}} which doesn't exist, so when submitted, the task errors out with a pydantic error as the parameters aren't present.

We can work around this by constructing the list of parameters via the names themselves, e.g. using a custom script constructor:

class InlineTemplateRunnerScriptConstructor(RunnerScriptConstructor):
    """A workaround constructor to allow our runner script to be an inline template."""

    def generate_source(self, instance: Script) -> str:
        """Convert inputs to a json string containing a list of dicts.

        Each dict contains the keys "name" and "value" with corresponding values
        for each input parameter in `instance`.
        """
        inputs = instance._build_inputs()
        if inputs is None or not inputs.parameters:
            return ""

        input_params = []
        for param in inputs.parameters:
            input_params.append(
                {"name": param.name, "value": f"{{{{inputs.parameters.{param.name}}}}}"}
            )

        return json.dumps(input_params)
@elliotgunton elliotgunton added the type:bug A general bug label May 17, 2023
flaviuvadan pushed a commit that referenced this issue Mar 25, 2024
Bumps [pytest-cov](https://github.com/pytest-dev/pytest-cov) from 4.1.0
to 5.0.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pytest-dev/pytest-cov/blob/master/CHANGELOG.rst">pytest-cov's
changelog</a>.</em></p>
<blockquote>
<h2>5.0.0 (2024-03-24)</h2>
<ul>
<li>Removed support for xdist rsync (now deprecated).
Contributed by Matthias Reichenbach in
<code>[#623](pytest-dev/pytest-cov#623)
&lt;https://github.com/pytest-dev/pytest-cov/pull/623&gt;</code>_.</li>
<li>Switched docs theme to Furo.</li>
<li>Various legacy Python cleanup and CI improvements.
Contributed by Christian Clauss and Hugo van Kemenade in
<code>[#630](pytest-dev/pytest-cov#630)
&lt;https://github.com/pytest-dev/pytest-cov/pull/630&gt;</code><em>,
<code>[#631](pytest-dev/pytest-cov#631)
&lt;https://github.com/pytest-dev/pytest-cov/pull/631&gt;</code></em>,
<code>[#632](pytest-dev/pytest-cov#632)
&lt;https://github.com/pytest-dev/pytest-cov/pull/632&gt;</code>_ and
<code>[#633](pytest-dev/pytest-cov#633)
&lt;https://github.com/pytest-dev/pytest-cov/pull/633&gt;</code>_.</li>
<li>Added a <code>pyproject.toml</code> example in the docs.
Contributed by Dawn James in
<code>[#626](pytest-dev/pytest-cov#626)
&lt;https://github.com/pytest-dev/pytest-cov/pull/626&gt;</code>_.</li>
<li>Modernized project's pre-commit hooks to use ruff. Initial POC
contributed by
Christian Clauss in
<code>[#584](pytest-dev/pytest-cov#584)
&lt;https://github.com/pytest-dev/pytest-cov/pull/584&gt;</code>_.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/5295ce01c84262cec88f31255e9ac538718f3047"><code>5295ce0</code></a>
Bump version: 4.1.0 → 5.0.0</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/1181b067972bf94569f8011f3b18f271690f9ab1"><code>1181b06</code></a>
Update changelog.</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/9757222e2e044361e70125ebdd96e5eb87395983"><code>9757222</code></a>
Fix a minor grammar error (<a
href="https://redirect.github.com/pytest-dev/pytest-cov/issues/636">#636</a>)</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/9f5cd81a0dbe3fe41681efdbef516c08988fe8ff"><code>9f5cd81</code></a>
Cleanup releasing instructions. Closes <a
href="https://redirect.github.com/pytest-dev/pytest-cov/issues/616">#616</a>.</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/93b5047ec5050d63c10a6fe16a09b671a7a03df8"><code>93b5047</code></a>
Add test for pyproject.toml loading without explicit --cov-config. Ref
<a
href="https://redirect.github.com/pytest-dev/pytest-cov/issues/508">#508</a>.</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/ff50860d7c67b920503745d92a3f0944cf41f982"><code>ff50860</code></a>
docs: add config instructions for pyproject.toml.</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/4a5a4b5fa4b1c63ddcab5cbc1813798c9b6f1d36"><code>4a5a4b5</code></a>
Keep GitHub Actions up to date with GitHub's Dependabot</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/1d7f55963d5138f41c452a946f7cca7e0b6ee8b2"><code>1d7f559</code></a>
Fix or remove URLs that are causing docs tests to fail</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/6a5af8e85b8242ac815f33e26adf9068f5f0ebc3"><code>6a5af8e</code></a>
Update changelog.</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/d9fe8dfed15023d3410dd299c5092e755b8981c2"><code>d9fe8df</code></a>
Switch to furo. Closes <a
href="https://redirect.github.com/pytest-dev/pytest-cov/issues/618">#618</a>.</li>
<li>Additional commits viewable in <a
href="https://github.com/pytest-dev/pytest-cov/compare/v4.1.0...v5.0.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pytest-cov&package-manager=pip&previous-version=4.1.0&new-version=5.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug A general bug
Projects
None yet
Development

No branches or pull requests

2 participants