Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

De-dupe release PURLs after reading them. #9

Merged
merged 2 commits into from
Sep 3, 2024
Merged

De-dupe release PURLs after reading them. #9

merged 2 commits into from
Sep 3, 2024

Conversation

tiegz
Copy link
Contributor

@tiegz tiegz commented Aug 30, 2024 •
edited
Loading

this should avoid waste during the Release lookup stage as well as the downloaded vuln report.

Demo using a big SBOM containing duplicates

Before

DEBU[0000] Found 590 purls                              
DEBU[0009] Getting vulnerability info for CVE-2024-21742 
DEBU[0009] Getting vulnerability info for CVE-2024-29857 
DEBU[0009] Getting vulnerability info for CVE-2024-34447 
DEBU[0009] Getting vulnerability info for CVE-2024-30172 
DEBU[0010] Getting vulnerability info for CVE-2024-29857 
DEBU[0010] Getting vulnerability info for CVE-2024-30171 
DEBU[0010] Getting vulnerability info for CVE-2024-26308 
DEBU[0010] Getting vulnerability info for CVE-2024-25710 
DEBU[0011] Getting vulnerability info for CVE-2024-23450 
DEBU[0011] Getting vulnerability info for CVE-2024-23451 
DEBU[0011] Getting vulnerability info for CVE-2024-23444 
DEBU[0011] Getting vulnerability info for CVE-2024-3651 
DEBU[0012] Getting vulnerability info for CVE-2023-35116 
DEBU[0012] Getting vulnerability info for CVE-2023-35116 
DEBU[0012] Getting vulnerability info for CVE-2023-35116 
DEBU[0012] Getting vulnerability info for CVE-2023-35116 
DEBU[0012] Getting vulnerability info for CVE-2023-35116 
DEBU[0013] Getting vulnerability info for CVE-2023-35116 
DEBU[0013] Getting vulnerability info for CVE-2023-35116 
DEBU[0013] Getting vulnerability info for CVE-2023-35116 
DEBU[0013] Getting vulnerability info for CVE-2023-35116 
DEBU[0013] Getting vulnerability info for CVE-2023-35116 
DEBU[0014] Getting vulnerability info for CVE-2023-1370 
DEBU[0014] Getting vulnerability info for CVE-2020-9488 
DEBU[0014] Getting vulnerability info for CVE-2024-29025 
DEBU[0014] Getting vulnerability info for CVE-2024-29025 
DEBU[0015] Getting vulnerability info for CVE-2024-29025 
DEBU[0015] Getting vulnerability info for CVE-2024-29025 
DEBU[0015] Getting vulnerability info for CVE-2024-29025 
DEBU[0015] Getting vulnerability info for CVE-2024-29025 
DEBU[0016] Getting vulnerability info for CVE-2024-29025 
DEBU[0016] Getting vulnerability info for CVE-2024-29025 
DEBU[0016] Getting vulnerability info for CVE-2024-29025 
DEBU[0016] Getting vulnerability info for CVE-2023-4586 
DEBU[0017] Getting vulnerability info for CVE-2023-4586 
DEBU[0017] Getting vulnerability info for CVE-2023-4586 
DEBU[0017] Getting vulnerability info for CVE-2023-4586 
DEBU[0017] Getting vulnerability info for CVE-2023-4586 
DEBU[0017] Getting vulnerability info for CVE-2023-4586 
DEBU[0018] Getting vulnerability info for CVE-2023-4586 
DEBU[0018] Getting vulnerability info for CVE-2023-4586 
DEBU[0018] Getting vulnerability info for CVE-2023-4586 
DEBU[0018] Getting vulnerability info for CVE-2023-52428 
DEBU[0019] Getting vulnerability info for CVE-2023-34062 
DEBU[0019] Getting vulnerability info for CVE-2023-32681 
DEBU[0019] Getting vulnerability info for CVE-2024-35195 
DEBU[0019] Getting vulnerability info for CVE-2022-40897 
DEBU[0019] Getting vulnerability info for CVE-2024-6345 
DEBU[0020] Getting vulnerability info for CVE-2024-23081 
DEBU[0020] Getting vulnerability info for CVE-2024-23082 
DEBU[0020] Getting vulnerability info for CVE-2023-45803 
DEBU[0020] Getting vulnerability info for CVE-2024-37891 
DEBU[0021] Getting vulnerability info for CVE-2023-43804

After

DEBU[0000] Found 462 purls                              
DEBU[0012] Getting vulnerability info for CVE-2023-35116 
DEBU[0012] Getting vulnerability info for CVE-2022-40897 
DEBU[0012] Getting vulnerability info for CVE-2024-6345 
DEBU[0012] Getting vulnerability info for CVE-2023-35116 
DEBU[0013] Getting vulnerability info for CVE-2023-32681 
DEBU[0013] Getting vulnerability info for CVE-2024-35195 
DEBU[0013] Getting vulnerability info for CVE-2023-34062 
DEBU[0013] Getting vulnerability info for CVE-2024-29025 
DEBU[0014] Getting vulnerability info for CVE-2023-52428 
DEBU[0014] Getting vulnerability info for CVE-2023-4586 
DEBU[0014] Getting vulnerability info for CVE-2024-21742 
DEBU[0015] Getting vulnerability info for CVE-2024-34447 
DEBU[0016] Getting vulnerability info for CVE-2024-30172 
DEBU[0016] Getting vulnerability info for CVE-2024-29857 
DEBU[0017] Getting vulnerability info for CVE-2024-30171 
DEBU[0017] Getting vulnerability info for CVE-2024-23450 
DEBU[0017] Getting vulnerability info for CVE-2024-23451 
DEBU[0018] Getting vulnerability info for CVE-2024-23444 
DEBU[0018] Getting vulnerability info for CVE-2023-1370 
DEBU[0018] Getting vulnerability info for CVE-2024-3651 
DEBU[0018] Getting vulnerability info for CVE-2020-9488 
DEBU[0019] Getting vulnerability info for CVE-2024-29857 
DEBU[0019] Getting vulnerability info for CVE-2024-26308 
DEBU[0019] Getting vulnerability info for CVE-2024-25710 
DEBU[0020] Getting vulnerability info for CVE-2024-23081 
DEBU[0020] Getting vulnerability info for CVE-2024-23082 
DEBU[0020] Getting vulnerability info for CVE-2023-45803 
DEBU[0020] Getting vulnerability info for CVE-2024-37891 
DEBU[0021] Getting vulnerability info for CVE-2023-43804

tiegz
tiegz commented Aug 30, 2024
View reviewed changes
go.mod
@@ -1,6 +1,6 @@
module github.com/tidelift/tidelift-sbom-info

go 1.22
go 1.23
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needed this at first for maps.Keys, but decided against it in the end. Here are the release notes: https://tip.golang.org/doc/go1.23

@tiegz tiegz requested a review from katzj August 30, 2024 23:39
@tiegz tiegz merged commit 831a223 into main Sep 3, 2024
2 checks passed
@tiegz tiegz deleted the tz/unique-purls branch September 3, 2024 04:32
@tiegz tiegz mentioned this pull request Sep 3, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@katzj katzj katzj approved these changes

Assignees
No one assigned
Labels
1 min review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tiegz @katzj