Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove noisy scenarios and add new scenario #327

Merged
merged 3 commits into from
Jul 26, 2024

Conversation

Anilm3
Copy link
Collaborator

@Anilm3 Anilm3 commented Jul 26, 2024

The current microbenchmark utility uses scenarios to test independent features, this suffers from unstable results due to many factors, such as code layout and alignment, WAF "runtime" changes, etc. This applies to all tests except the global ruleset one.
To simplify things and potentially resolve the constant noise from benchmarks, the current scenario based benchmark will only be used as a macrobenchmark which will contain only wide-ranging scenarios, starting with a single, full ruleset one.

Note that the benchmarks will fail as no scenarios are common to the master branch.

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.52%. Comparing base (1a67a2b) to head (3948fbb).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #327   +/-   ##
=======================================
  Coverage   84.52%   84.52%           
=======================================
  Files         144      144           
  Lines        6941     6941           
  Branches     3179     3179           
=======================================
  Hits         5867     5867           
  Misses        400      400           
  Partials      674      674           
Flag Coverage Δ
waf_test 84.52% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Anilm3 Anilm3 marked this pull request as ready for review July 26, 2024 13:16
@Anilm3 Anilm3 requested a review from a team as a code owner July 26, 2024 13:16
@Anilm3 Anilm3 merged commit dd8feb4 into master Jul 26, 2024
49 of 50 checks passed
@Anilm3 Anilm3 deleted the anilm3/simplify-benchmarks branch July 26, 2024 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants