Rework chunk cache #6773
Merged
Rework chunk cache #6773
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## develop #6773 +/- ##
===========================================
- Coverage 82.46% 82.45% -0.01%
===========================================
Files 366 366
Lines 39761 39761
Branches 3545 3545
===========================================
- Hits 32787 32784 -3
- Misses 6974 6977 +3
|
azhavoro
requested review from
SpecLad
and removed request for
mdacoca and
Marishka17
|
Please rebase, as this conflicts with #6761. |
SpecLad
requested changes
Sep 1, 2023
SpecLad
reviewed
Sep 4, 2023
SpecLad
reviewed
Sep 6, 2023
| "/etc/keydb/keydb.conf", | ||
| "--storage-provider", "flash", "/data/flash", | ||
| "--maxmemory", "5G", | ||
| "--maxmemory-policy", "allkeys-lfu", |
There was a problem hiding this comment.
We're using the same KeyDB instance for both asynchronous tasks and caching. For cached objects, this policy makes sense, but wouldn't it cause problems for task queues? We could start randomly losing background tasks if their DB entries get evicted.
There was a problem hiding this comment.
https://docs.keydb.dev/docs/flash:
With KeyDB on FLASH you can set maxmemory to the amount of DRAM you want allocated without the fear of losing data when exceeded. Instead the data is removed from DRAM but remains in FLASH where it is not lost and can still be served.
There was a problem hiding this comment.
Ah, alright. That's reassuring.
SpecLad
approved these changes
Sep 6, 2023
Merged
nmanovic
added a commit
that referenced
this pull request
Sep 6, 2023
### Added - Gamma correcton filter (<#6771>) - Introduced the feature to hide or show objects in review mode (<#6808>) ### Changed - \[Helm\] Database migrations are now executed as a separate job, rather than in the server pod, to mitigate the risk of data corruption when using multiple server replicas (<#6780>) - Clicking multiple times on icons in the left sidebar now toggles the corresponding popovers open and closed (<#6817>) - Transitioned to using KeyDB with FLASH for data chunk caching, replacing diskcache (<#6773>) ### Removed - Removed outdated use of hostnames when accessing Git, OpenCV, or analytics via the UI (<#6799>) - Removed the Feedback/Share component (<#6805>) ### Fixed - Resolved the issue of the canvas zooming while scrolling through the comments list in an issue (<#6758>) - Addressed the bug that allowed for multiple issue creations upon initial submission (<#6758>) - Fixed the issue of running deep learning models on non-JPEG compressed TIFF images (<#6789>) - Adjusted padding on the tasks, projects, and models pages (<#6778>) - Corrected hotkey handlers to avoid overriding default behavior when modal windows are open (<#6800>) - Resolved the need to move the mouse to activate brush or eraser effects; a single click is now sufficient (<#6800>) - Fixed a memory leak issue in the logging system (<#6804>) - Addressed a race condition that occurred during the initial creation of `secret_key.py` (<#6775>) - Eliminated duplicate log entries generated by the CVAT server (<#6766>)
mikhail-treskin
pushed a commit
to retailnext/cvat
that referenced
this pull request
Oct 25, 2023
<!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [x] I have added a description of my changes into the [CHANGELOG](https://github.com/opencv/cvat/blob/develop/CHANGELOG.md) file - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. --------- Co-authored-by: Roman Donchenko <roman@cvat.ai>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and context
How has this been tested?
Checklist
developbranch(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)
License
Feel free to contact the maintainers if that's a concern.