Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[10.4-stable] backport: httputil: fix potential memleak #3713

Conversation

christoph-zededa
Copy link
Contributor

@christoph-zededa christoph-zededa commented Jan 25, 2024

IMHO similar to 9c194dd the memleak also has to be fixed here

The fix for a newer version has also been introduced here: lf-edge/eve-libs@127266b#diff-4a3ca89f1c1037aa18b0258facf633f93de2016bfb09906794e7cc7c88f4fd62R264 (which has then been replaced by lf-edge/eve-libs@08a38c8 )

IMHO similar to 9c194dd the memleak
also has to be fixed here

Signed-off-by: Christoph Ostarek <christoph@zededa.com>
@OhmSpectator
Copy link
Member

We'll try to reproduce/verify with a huge application used to trigger a mem leak before.

@OhmSpectator
Copy link
Member

@rene, @christoph-zededa, @milan-zededa, so, I guess, we should switch it to the "ready to review" state? )

copied from lfedge/eve-libs:307499b404872ab07d0ec20c59cdf7a8c7cd6936

Signed-off-by: Christoph Ostarek <christoph@zededa.com>
@christoph-zededa
Copy link
Contributor Author

I added another commit, porting lf-edge/eve-libs@307499b to here

@OhmSpectator
Copy link
Member

The second leak! =)
Can we merge it now, or do we expect several more? =)

@christoph-zededa christoph-zededa marked this pull request as ready for review January 26, 2024 18:57
Copy link
Contributor

@rouming rouming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This indeed fixes the described leaks, but in order to see inactivityTimer leak in action you need to request download of thousands of images in 5 minute, which seems is not the case. @christoph-zededa please backport the inactivityTimer fix to all the branches to keep the code same.

The socket leak is nasty, but I doubt you hit the memory pressure before you hit the limit of number of opened file descriptors (which I assume not a huge number, at least I can't find that we call the setrlimit(RLIMIT_NOFILE) in the code)

Anyway, good catch

@christoph-zededa christoph-zededa changed the title httputil: fix potential memleak [10.4-stable] backport: httputil: fix potential memleak Jan 29, 2024
@christoph-zededa christoph-zededa marked this pull request as draft January 29, 2024 10:17
@christoph-zededa
Copy link
Contributor Author

Converting to draft, first have to find out how to do a proper make proto-vendor

Did:

cp libs/zedUpload/httputil/http.go pkg/pillar/vendor/github.com/lf-edge/eve/libs/zedUpload/httputil/http.go

as make proto && make proto-vendor does not have an impact

Signed-off-by: Christoph Ostarek <christoph@zededa.com>
@christoph-zededa christoph-zededa marked this pull request as ready for review January 29, 2024 10:55
@christoph-zededa
Copy link
Contributor Author

Converting to draft, first have to find out how to do a proper make proto-vendor

Please see 59677cd

@christoph-zededa
Copy link
Contributor Author

This indeed fixes the described leaks, but in order to see inactivityTimer leak in action you need to request download of thousands of images in 5 minute, which seems is not the case. @christoph-zededa please backport the inactivityTimer fix to all the branches to keep the code same.

inactivityTimer is only created once for a download attempt, while the new TimeoutReader creates a new Timer on every Read

Backport for 9.4-stable is here: #3722

11.0-stable go.mod is here: https://github.com/lf-edge/eve/blob/11.0-stable/pkg/pillar/go.mod#L28
it points to 666ed2307f8e which is the current head of https://github.com/lf-edge/eve-libs and therefore includes the fix.

Copy link
Member

@OhmSpectator OhmSpectator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But shouldn't we fix the version in go.mod as well? Not it points to github.com/lf-edge/eve/libs v0.0.0-20230717153241-cc5629af4657 and if I run go mod vendor it rolls the lib file in the vendor directory back to the older version...
Sorry if I'm asking a stupid question, I'm still confused with the go modules stuff.

@OhmSpectator
Copy link
Member

OhmSpectator commented Jan 29, 2024

Argh... Discussed it with @christoph-zededa offline. As I understand it now, the algorithm is the following:

  1. Merge this PR
  2. Get a hash of the merged commit that fixes the lib
  3. Put the hash into go.mod
  4. Create a new PR

I remember we discussed it before with @milan-zededa and @deitch, but I'm always forgetting that...

Copy link
Member

@OhmSpectator OhmSpectator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, we have to merge it and get a new PR with go.mod update.

Copy link
Contributor

@eriknordmark eriknordmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eriknordmark eriknordmark merged commit b795a06 into lf-edge:10.4-stable Jan 31, 2024
17 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants