[13_0_X] Change of GE21 layouts and plots, and VFAT status with VFAT mask and missing VFATs

[quark2]

PR description:

In this PR, plots for GE21 have been divided into modules (+layers). In the same manner, the cells for GE21 in the summary plot have been rearranged by module(+layer).

Also, masked VFATs and error/warning status of missing VFATs are now being displayed.

This is a backport of #41915 to 13_0_X.

PR validation:

Tests are done with cmsRun $CMSSW_RELEASE_BASE/src/DQM/Integration/python/clients/gem_dqm_sourceclient-live_cfg.py unitTest=True runNumber=362760 dataset=/ExpressPhysics/Run2022G-Express-v1/FEVT minLumi=19 maxLumi=21 and runTheMatrix.py -l limited -i all --ibeos since it makes effects on P5

@jshlee @watson-ij @seungjin-yang @yeckang

[cmsbuild]

A new Pull Request was created by @quark2 for CMSSW_13_0_X.

It involves the following packages:

• DQM/GEM (dqm)
• Validation/MuonGEMHits (dqm)

@nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@jshlee, @missirol, @JanFSchulte, @watson-ij this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from Change of GE21 layouts and plots, and VFAT status with VFAT mask and missing VFATs #41915

[perrotta]

backport of #41915

[perrotta]

please test

[perrotta]

@quark2 please provide in addition at least the backport to the next data taking release cycle, 13_2_X.

[quark2]

Hi @perrotta,

Okay, I'll make the backport soon. How about a backport to 13_1_X?

[quark2]

Ahh, never mind. I was told I don't need to consider 13_1_X.

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79fb6f/34114/summary.html
COMMIT: eac6a3d
CMSSW: CMSSW_13_0_X_2023-08-06-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42482/34114/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

• You potentially removed 227 lines from the logs
• Reco comparison results: 6 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3280689
• DQMHistoTests: Total failures: 27
• DQMHistoTests: Total nulls: 56
• DQMHistoTests: Total successes: 3280584
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 4680.928000000001 KiB( 48 files compared)
• DQMHistoSizes: changed ( 11634.0,... ): 9.615 KiB GEM/Digis
• DQMHistoSizes: changed ( 11634.0,... ): -1.617 KiB GEM/DAQStatus
• DQMHistoSizes: changed ( 11634.0,... ): -0.161 KiB GEM/EventInfo
• DQMHistoSizes: changed ( 20834.0,... ): 664.605 KiB GEM/Digis
• DQMHistoSizes: changed ( 20834.0,... ): -3.938 KiB GEM/DAQStatus
• DQMHistoSizes: changed ( 20834.0,... ): 0.199 KiB GEM/EventInfo
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[emanueleusai]

type gem

[emanueleusai]

please test

[emanueleusai]

@quark2 do you need this in online DQM machines?

[quark2]

Hi @emanueleusai,

Yep, I need it in the DQM machine.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79fb6f/34127/summary.html
COMMIT: eac6a3d
CMSSW: CMSSW_13_0_X_2023-08-07-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42482/34127/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 8 lines from the logs
• Reco comparison results: 10 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3280689
• DQMHistoTests: Total failures: 30
• DQMHistoTests: Total nulls: 56
• DQMHistoTests: Total successes: 3280581
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 4680.928000000001 KiB( 48 files compared)
• DQMHistoSizes: changed ( 11634.0,... ): 9.615 KiB GEM/Digis
• DQMHistoSizes: changed ( 11634.0,... ): -1.617 KiB GEM/DAQStatus
• DQMHistoSizes: changed ( 11634.0,... ): -0.161 KiB GEM/EventInfo
• DQMHistoSizes: changed ( 20834.0,... ): 664.605 KiB GEM/Digis
• DQMHistoSizes: changed ( 20834.0,... ): -3.938 KiB GEM/DAQStatus
• DQMHistoSizes: changed ( 20834.0,... ): 0.199 KiB GEM/EventInfo
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[emanueleusai]

testing at p5

[emanueleusai]

+1

• p5 tests ok

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1

[perrotta]

By the way, there is backport to 13_0_X and 13_2_X but not to 13_1_X: I imagine that it is not needed by @cms-sw/dqm-l2 in that intermediate release intended for Phase2