Adds undercover gem to the stack

[filipefurtad0]

What? Why?

(Please note: do not try to install this gem with a conda environment running ❗ )

Adds undercover gem to the stack and runs it within every CI node - by adding a Check code coverage changes task:

If there are no changes, we should see:

This, however, is not seen on the first pic. I think the rake task is not doing what it should - it should yield the same output as when ran locallt. Any ideas here? ❓

What should we test?

There are two main use cases for this gem:

• to find untested code changes locally (with the CLI)

To do this locally, run:
bundle exec undercover --compare origin/master

• to have review checks on GitHub

To achieve this, a a rake task was created and called within each type on specs, within build.yml. I'm not sure this is the best approach, happy for comments/suggestions/improvements!

Ideally, we'd test both scenarios, locally and remotely (CI):

Locally

• Commiting a code change without adding tests, on a given branch
• Run bundle exec undercover --compare origin/master
• One should see an output from indicating changes on code coverage

Remotely/CI

• Push the previous commits
• On each node, we should be able to see an output, indicating code changes

Release notes

Changelog Category (reviewers may add a label for the release notes):

• User facing changes
• API changes (V0, V1, DFC or Webhook)
• Technical changes only
• Feature toggled

The title of the pull request will be included in the release notes.

Dependencies

Documentation updates

[filipefurtad0]

Requesting early review on this one. I can't seem to progress much more here. Issues I've found:

• we get this warning, when running undercover on CI: parser/current is loading parser/ruby31, which recognizes 3.1.6-compliant syntax, but you are running 3.1.4.
• I've deleted (and committed) specs, and ran undercover locally. I was expecting to see some warning on code coverage reduction (this is the opposite of adding uncovered code, but I figured it should work). Another approach could be to cherry pick some commits with no tests, from other branches

[filipefurtad0]

I'm not sure this is a good start. If this takes too much capacity or is a too curly effort at this stage, please feel free to close this PR. Thanks! 🙏

[mkllnk]

I think in order to compare to master, it first needs a coverage report for master. So we would need to run it on master and commit those files. Or are they uploaded somewhere? I think that it's unrealistic to keep the coverage report on master up-to-date unless it's automated. And that's really difficult with a parallelised build like we have. That's were other simplecov integrations failed in the past.

Oh, I see, we need to sign up at https://undercover-ci.com/ and it's free for open source. It could work but the parallel build may make it unreliable. Not sure if it's worth it.

[mkllnk]

What is the advantage of a rake task?

[filipefurtad0]

I tried to follow the syntax or way we run tasks from build.yml, we seem to have opted for running them with bin/rake, for example:

        run: |
          bin/rake knapsack_pro:rspec

[mkllnk]

Ah, yes. I think that's Knapsack specific. Before Knapsack, we used to run bundle exec rspec.

[mkllnk]

That's an odd condition. We set the rails env above and then it's always set. The condition is always true and therefore useless.

[filipefurtad0]

Agree. Perhaps it would be better to remove if ENV["COVERAGE"] when requiring 'simplecov', I think we'd need it to run undercover locally.

[dacook]

Sounds like there's still a bit to work out, so maybe it's worth parking this?
It could be closed, but still viewable now that I've attached it to the issue #11948

[filipefurtad0]

It could be closed, but still viewable now that I've attached it to the issue #11948

Agree, I think it's best to close it. Thank you for linking it to the issue @dacook 🙏