Fix log message duplication #6766
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SpecLad
force-pushed
the
duplicate-logs
branch
from
fe24c78
to
2032c63
Compare
Codecov Report
@@ Coverage Diff @@
## develop #6766 +/- ##
===========================================
- Coverage 82.46% 82.45% -0.01%
===========================================
Files 366 366
Lines 39761 39761
Branches 3545 3545
===========================================
- Hits 32787 32784 -3
- Misses 6974 6977 +3
|
There are two independent problems that cause some log messages to be
printed twice:
* In the CVAT logging configuration, we attach a console handler to the
`cvat.server` and `django` loggers. However, when any library[1] used by
CVAT uses one of the top-level `logging` functions (e.g. `logging.info`),
then it will cause the root logger to get autoconfigured with another
console handler. Since `cvat.server` and `django` are configured to
propagate events to the ancestor logger's handlers, those events are
passed to _two_ console handlers, so they are printed twice.
Fix it by moving our handlers to the root logger. If the root logger has
handlers, it will no longer get autoconfigured when top-level `logging`
functions are invoked; so there will only be one console handler in the
logger chain.
An alternative would be to configure the root logger with a `NullHandler`
(which will also disable autoconfiguration), but this will suppress all
logs that are coming from places other than Django and CVAT, which seems
undesirable.
* The supervisord config file for the server redirects the server's stdout
to supervisord's stdout. However, supervisord already displays the stdout
of all child processes due to the `loglevel=debug` setting. So all stdout
messages of the server are displayed twice.
Fix it by removing the redirection.
[1] Specifically, Datumaro does this. It could probably be fixed, but that
wouldn't fix the root problem, which is the duplicate console handler.
SpecLad
force-pushed
the
duplicate-logs
branch
from
2032c63
to
ca4164d
Compare
|
@azhavoro Could you review this? |
|
LGTM |
azhavoro
approved these changes
Sep 6, 2023
Merged
nmanovic
added a commit
that referenced
this pull request
Sep 6, 2023
### Added - Gamma correcton filter (<#6771>) - Introduced the feature to hide or show objects in review mode (<#6808>) ### Changed - \[Helm\] Database migrations are now executed as a separate job, rather than in the server pod, to mitigate the risk of data corruption when using multiple server replicas (<#6780>) - Clicking multiple times on icons in the left sidebar now toggles the corresponding popovers open and closed (<#6817>) - Transitioned to using KeyDB with FLASH for data chunk caching, replacing diskcache (<#6773>) ### Removed - Removed outdated use of hostnames when accessing Git, OpenCV, or analytics via the UI (<#6799>) - Removed the Feedback/Share component (<#6805>) ### Fixed - Resolved the issue of the canvas zooming while scrolling through the comments list in an issue (<#6758>) - Addressed the bug that allowed for multiple issue creations upon initial submission (<#6758>) - Fixed the issue of running deep learning models on non-JPEG compressed TIFF images (<#6789>) - Adjusted padding on the tasks, projects, and models pages (<#6778>) - Corrected hotkey handlers to avoid overriding default behavior when modal windows are open (<#6800>) - Resolved the need to move the mouse to activate brush or eraser effects; a single click is now sufficient (<#6800>) - Fixed a memory leak issue in the logging system (<#6804>) - Addressed a race condition that occurred during the initial creation of `secret_key.py` (<#6775>) - Eliminated duplicate log entries generated by the CVAT server (<#6766>)
mikhail-treskin
pushed a commit
to retailnext/cvat
that referenced
this pull request
Oct 25, 2023
There are two independent problems that cause some log messages to be printed twice: * In the CVAT logging configuration, we attach a console handler to the `cvat` and `django` loggers. However, when any library[1] used by CVAT uses one of the top-level `logging` functions (e.g. `logging.info`), then it will cause the root logger to get autoconfigured with another console handler. Since `cvat` and `django` are configured to propagate events to the ancestor logger's handlers, those events are passed to _two_ console handlers, so they are printed twice. Fix it by moving our handlers to the root logger. If the root logger has handlers, it will no longer get autoconfigured when top-level `logging` functions are invoked; so there will only be one console handler in the logger chain. An alternative would be to configure the root logger with a `NullHandler` (which will also disable autoconfiguration), but this will suppress all logs that are coming from places other than Django and CVAT, which seems undesirable. * The supervisord config file for the server redirects the server's stdout to supervisord's stdout. However, supervisord already displays the stdout of all child processes due to the `loglevel=debug` setting. So all stdout messages of the server are displayed twice. Fix it by removing the redirection. [1] Specifically, Datumaro does this. It could probably be fixed, but that wouldn't fix the root problem, which is the duplicate console handler.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and context
There are two independent problems that cause some log messages to be printed twice:
In the CVAT logging configuration, we attach a console handler to the
cvatanddjangologgers. However, when any library[1] used by CVAT uses one of the top-levelloggingfunctions (e.g.logging.info), then it will cause the root logger to get autoconfigured with another console handler. Sincecvatanddjangoare configured to propagate events to the ancestor logger's handlers, those events are passed to two console handlers, so they are printed twice.Fix it by moving our handlers to the root logger. If the root logger has handlers, it will no longer get autoconfigured when top-level
loggingfunctions are invoked; so there will only be one console handler in the logger chain.An alternative would be to configure the root logger with a
NullHandler(which will also disable autoconfiguration), but this will suppress all logs that are coming from places other than Django and CVAT, which seems undesirable.The supervisord config file for the server redirects the server's stdout to supervisord's stdout. However, supervisord already displays the stdout of all child processes due to the
loglevel=debugsetting. So all stdout messages of the server are displayed twice.Fix it by removing the redirection.
[1] Specifically, Datumaro does this. It could probably be fixed, but that wouldn't fix the root problem, which is the duplicate console handler.
How has this been tested?
Checklist
developbranch[ ] I have updated the documentation accordingly[ ] I have added tests to cover my changes[ ] I have linked related issues (see GitHub docs)[ ] I have increased versions of npm packages if it is necessary(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)
License
Feel free to contact the maintainers if that's a concern.