clustersynchro: handle each of the pages in the resource list stage #591
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind feature
What this PR does / why we need it:
In List, the paged pull method is used, so the resources within each paging will be processed directly.
Added two FeatureGates:
StreamHandlePaginatedListForResourceSync
: stream handling of paged resources, resources within a pager will be processed as soon as possible instead of waiting until all resources are pulled before calling the ResourceHandler.ForcePaginatedListForResourceSync
: force paging, Reflector will sometimes use APIServer's cache, even if paging is specified APIServer will return all resources for performance, then it will skip Reflector's streaming memory optimization.ForcePaginatedListForResourceSync
may put extra pressure on the imported cluster APIServer, please be careful to enable it.clustersynchro-manager add
--page-size
flags, you can control the page size of the List, the smaller the page size, the smaller the memory consumption, but may also generate more requests to APIServer.Which issue(s) this PR fixes:
fix: #594
Special notes for your reviewer:
Does this PR introduce a user-facing change?: