perf: collisions don't contain reserved names by default #684
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The 'collisions' set does NOT contain RESERVED_NAMES; they are combined at runtime when needed. For a large real-world, this results in an order of magnitude reduction in memory usage. I'm not joking: Google Ads v5 uses 2.45 GB before this change and 223 MB after.
software-dov
requested review from
lukesneeringer,
crwilcox,
vchudnov-g,
busunkim96 and
xiaozhenliu-gg5
google-cla
bot
added
the
cla: yes
Codecov Report
@@ Coverage Diff @@
## master #684 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 26 26
Lines 1561 1560 -1
Branches 315 316 +1
=========================================
- Hits 1561 1560 -1
Continue to review full report at Codecov.
|
busunkim96
approved these changes
Oct 27, 2020
There was a problem hiding this comment.
Also contains changes to add slots attributes to Metadata and Address. These are ancillary, optional, and open to negotiation.
In the above scenario, they reduce memory usage from 223 MB to 177 MB. If other people feel that the reduction in readability does not warrant the reduction in memory usage I am absolutely open to dropping that particular commit.
I would personally prefer preserving readability.
gapic/schema/metadata.py
Outdated
Comment on lines
151
to
152
| # Don't keep RESERVED_NAMES in self.collisions, | ||
| # just combine the two when necessary. |
There was a problem hiding this comment.
Perhaps add a note that this is for memory usage?
software-dov
force-pushed
the
collisions
branch
from
9e25710
to
50c4dcd
Compare
software-dov
added
the
automerge
gcf-merge-on-green
bot
removed
the
automerge
gcf-merge-on-green bot
pushed a commit
that referenced
this pull request
Oct 27, 2020
🤖 I have created a release \*beep\* \*boop\* --- ### [0.35.9](https://www.github.com/googleapis/gapic-generator-python/compare/v0.35.8...v0.35.9) (2020-10-27) ### Performance Improvements * collisions don't contain reserved names by default ([#684](https://www.github.com/googleapis/gapic-generator-python/issues/684)) ([2ec6ea6](https://www.github.com/googleapis/gapic-generator-python/commit/2ec6ea6835256c0d7b252e035cf4eac1ff442647)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please).
This was referenced Jan 18, 2022
This was referenced Jan 26, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The 'collisions' set does NOT contain RESERVED_NAMES; they are
combined at runtime when needed.
For a large real-world, this results in an order of magnitude
reduction in memory usage.
I'm not joking: Google Ads v5 uses 2.45 GB peak before this change
and 223 MB after.
Also contains changes to add
__slots__attributes to Metadata and Address. These are ancillary, optional, and open to negotiation.In the above scenario, they reduce memory usage from 223 MB to 177 MB. If other people feel that the reduction in readability does not warrant the reduction in memory usage I am absolutely open to dropping that particular commit.
Includes other minor memory usage optimizations that collectively shave about 5 MB.