[13_0_X] Addition of 2023 WFs and Fixing NANO step (Fix typo) #42191

srimanob · 2023-07-05T12:10:30Z

PR description:

This is a clone of #42089 with typo-fix, MINIAOOD => MiniAOD [here]

PR validation:

Run wf 140.001 which failed in IB before.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This is a backport PR.

cmsbuild · 2023-07-05T12:10:56Z

A new Pull Request was created by @srimanob (Phat Srimanobhas) for CMSSW_13_0_X.

It involves the following packages:

Configuration/PyReleaseValidation (pdmv, upgrade)

@cmsbuild, @bbilin, @srimanob, @sunilUIET, @AdrianoDee can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @Martin-Grunewald, @missirol, @fabiocos, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from addition of 2023C WFs and modifications 2023B WFs to run with era Run3 #42069

srimanob · 2023-07-05T12:15:10Z

test parameters:

workflows = 140.001,140.002,140.004,140.005,140.115,140.116
relvals_opt = --what cleanedupgrade,standard,highstats,pileup,generator,extendedgen,production,identity,ged,machine,premix,nano,gpu,2017,2026

srimanob · 2023-07-05T12:16:09Z

@cmsbuild please test

cmsbuild · 2023-07-05T15:39:29Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ba4325/33561/summary.html
COMMIT: 82f5d21
CMSSW: CMSSW_13_0_X_2023-07-05-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42191/33561/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 92 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 4281 differences found in the comparisons
DQMHistoTests: Total files compared: 53
DQMHistoTests: Total histograms compared: 3492431
DQMHistoTests: Total failures: 1176
DQMHistoTests: Total nulls: 13
DQMHistoTests: Total successes: 3491220
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -474403.463 KiB( 52 files compared)
DQMHistoSizes: changed ( 140.001,... ): -41173.651 KiB HLT/JME
DQMHistoSizes: changed ( 140.001,... ): -12323.616 KiB HLT/Tracking
DQMHistoSizes: changed ( 140.001,... ): -10390.469 KiB HLT/SiStrip
DQMHistoSizes: changed ( 140.001,... ): -10083.229 KiB HLT/EXO
DQMHistoSizes: changed ( 140.001,... ): -8697.605 KiB HLT/Pixel
DQMHistoSizes: changed ( 140.001,... ): -3626.419 KiB HLT/TOP
DQMHistoSizes: changed ( 140.001,... ): -3575.395 KiB HLT/EGM
DQMHistoSizes: changed ( 140.001,... ): -3336.544 KiB HLT/HIG
DQMHistoSizes: changed ( 140.001,... ): -2827.491 KiB HLT/SUSY
DQMHistoSizes: changed ( 140.001,... ): -1593.688 KiB HLT/Objects
DQMHistoSizes: changed ( 140.001 ): ...
Checked 237 log files, 184 edm output root files, 53 DQM output files
TriggerResults: found differences in 4 / 51 workflows

srimanob · 2023-07-05T16:44:35Z

+Upgrade

A clone of #42089 with typo fix. Change is expected due to HLT step.

sunilUIET · 2023-07-06T04:45:41Z

+pdmv

cmsbuild · 2023-07-06T04:46:03Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta

Since this already missed the deadline for the CMSSW_13_0_10 release, I would profit and fix also the comment line, in order to avoid possible problems when it will get uncommented.

Configuration/PyReleaseValidation/python/relval_standard.py

perrotta · 2023-07-06T05:57:39Z

backport of #42069
(Overall:

partial backport of addition of 2023C WFs and modifications 2023B WFs to run with era Run3 #42069
partial backport of Add 2023 data relval workflows #41629
backport of the bug fixes in [Bug-fix] Add NANOAOD output to RECONANORUN3 step #42047)

)

cmsbuild · 2023-07-06T09:10:34Z

Pull request #42191 was updated. @cmsbuild, @bbilin, @srimanob, @sunilUIET, @AdrianoDee can you please check and sign again.

srimanob · 2023-07-06T09:11:14Z

@cmsbuild please test

slava77 · 2023-07-06T12:22:54Z

Configuration/PyReleaseValidation/python/relval_standard.py

-workflows[140.116] = ['',['RunCommissioning2022D','HLTDR3_2023','RECONANORUN3_reHLT','SKIMCOMMISSIONINGRUN3_reHLT']]
-workflows[140.117] = ['',['RunCosmics2022D','HLTDR3_2023','RECOCOSMRUN3_reHLT','SKIMCOSMICSRUN3_reHLT']]
-#workflows[140.118] = ['',['RunParkingBPH2022B','HLTDR3_2023','RECONANORUN3_reHLT','SKIMPARKINGBPHRUN3_reHLT']]
+workflows[140.104] = ['',['RunZeroBias2022D','HLTDR3_2022','RECONANORUN3_reHLT','SKIMZEROBIASRUN3_reHLT']]


we need a ZB flavor

we need a ZB flavor

ah, there is 140.063] = ['',['RunZeroBias2022D'.
Is this just a no-DQM production like setup?

Good question @slava77

Looking in master, this workflow produce DQM from RECONANORUN3_ZB_reHLT_2022 (or RECONANORUN3_reHLT in 13_0). But there is no harvesting step.

@cms-sw/pdmv-l2 Is this your intention, or missing?

master

@srimanob @slava77 I am not sure of the use case of WFs 140.104-140.118. For consistency, I changed the DQM sequence to ZeroBias 140.104 as well but missed the changes in the Harvesting step. As the ZeroBias DQM request is initially from tracking @mmusich, as per their need either we can revert back to standard DQM or update harvesting step for this WF.

cmsbuild · 2023-07-06T13:34:13Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ba4325/33568/summary.html
COMMIT: ca6223e
CMSSW: CMSSW_13_0_X_2023-07-05-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42191/33568/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

You potentially removed 95 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 4281 differences found in the comparisons
DQMHistoTests: Total files compared: 53
DQMHistoTests: Total histograms compared: 3492431
DQMHistoTests: Total failures: 1176
DQMHistoTests: Total nulls: 13
DQMHistoTests: Total successes: 3491220
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -474403.463 KiB( 52 files compared)
DQMHistoSizes: changed ( 139.001 ): -41169.464 KiB HLT/JME
DQMHistoSizes: changed ( 139.001 ): -7299.500 KiB HLT/EXO
DQMHistoSizes: changed ( 139.001,... ): -3626.419 KiB HLT/TOP
DQMHistoSizes: changed ( 139.001 ): -3271.642 KiB HLT/HIG
DQMHistoSizes: changed ( 139.001,... ): -2827.491 KiB HLT/SUSY
DQMHistoSizes: changed ( 139.001 ): -1827.372 KiB HLT/BTV
DQMHistoSizes: changed ( 139.001 ): -1721.552 KiB HLT/EGM
DQMHistoSizes: changed ( 139.001 ): -1593.265 KiB HLT/Objects
DQMHistoSizes: changed ( 139.001 ): -1505.974 KiB HLT/B2G
DQMHistoSizes: changed ( 139.001 ): -366.666 KiB HLT/Filters
DQMHistoSizes: changed ( 139.001 ): ...
Checked 237 log files, 184 edm output root files, 53 DQM output files
TriggerResults: found differences in 4 / 51 workflows

srimanob · 2023-07-07T13:02:03Z

@cmsbuild please test

cmsbuild · 2023-07-07T16:03:55Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ba4325/33595/summary.html
COMMIT: ca6223e
CMSSW: CMSSW_13_0_X_2023-07-07-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42191/33595/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 113 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 4398 differences found in the comparisons
DQMHistoTests: Total files compared: 53
DQMHistoTests: Total histograms compared: 3492431
DQMHistoTests: Total failures: 5282
DQMHistoTests: Total nulls: 13
DQMHistoTests: Total successes: 3487114
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -474403.463 KiB( 52 files compared)
DQMHistoSizes: changed ( 140.001,... ): -41173.651 KiB HLT/JME
DQMHistoSizes: changed ( 140.001,... ): -12323.616 KiB HLT/Tracking
DQMHistoSizes: changed ( 140.001,... ): -10390.469 KiB HLT/SiStrip
DQMHistoSizes: changed ( 140.001,... ): -10083.229 KiB HLT/EXO
DQMHistoSizes: changed ( 140.001,... ): -8697.605 KiB HLT/Pixel
DQMHistoSizes: changed ( 140.001,... ): -3626.419 KiB HLT/TOP
DQMHistoSizes: changed ( 140.001,... ): -3575.395 KiB HLT/EGM
DQMHistoSizes: changed ( 140.001,... ): -3336.544 KiB HLT/HIG
DQMHistoSizes: changed ( 140.001,... ): -2827.491 KiB HLT/SUSY
DQMHistoSizes: changed ( 140.001,... ): -1593.688 KiB HLT/Objects
DQMHistoSizes: changed ( 140.001 ): ...
Checked 237 log files, 184 edm output root files, 53 DQM output files
TriggerResults: found differences in 4 / 51 workflows

srimanob · 2023-07-07T17:25:19Z

+Upgrade

I would propose to merge this PR as is, and fix for harvesting in different PRs, follow the fix in master.

sunilUIET · 2023-07-08T03:52:00Z

+pdmv

cmsbuild · 2023-07-08T03:52:29Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-07-08T11:25:02Z

+1

42089 with typo fix

82f5d21

cmsbuild added this to the CMSSW_13_0_X milestone Jul 5, 2023

cmsbuild added pending-signatures tests-pending orp-pending pdmv-pending upgrade-pending labels Jul 5, 2023

srimanob mentioned this pull request Jul 5, 2023

[13_0_X] Addition of 2023 WFs and Fixing NANO step #42089

Merged

cmsbuild added tests-started and removed tests-pending labels Jul 5, 2023

cmsbuild added tests-approved and removed tests-started labels Jul 5, 2023

cmsbuild added upgrade-approved and removed upgrade-pending labels Jul 5, 2023

cmsbuild added fully-signed pdmv-approved and removed pending-signatures pdmv-pending labels Jul 6, 2023

perrotta reviewed Jul 6, 2023

View reviewed changes

Configuration/PyReleaseValidation/python/relval_standard.py Outdated Show resolved Hide resolved

cmsbuild added the backport-ok label Jul 6, 2023

fix 140.118

ca6223e

cmsbuild removed the fully-signed label Jul 6, 2023

cmsbuild added tests-pending pdmv-pending upgrade-pending labels Jul 6, 2023

cmsbuild added tests-started and removed tests-pending labels Jul 6, 2023

slava77 reviewed Jul 6, 2023

View reviewed changes

cmsbuild added tests-rejected and removed tests-started labels Jul 6, 2023

cmsbuild added tests-started and removed tests-rejected labels Jul 7, 2023

cmsbuild added tests-approved and removed tests-started labels Jul 7, 2023

cmsbuild added upgrade-approved and removed upgrade-pending labels Jul 7, 2023

cmsbuild added fully-signed pdmv-approved and removed pending-signatures pdmv-pending labels Jul 8, 2023

cmsbuild added orp-approved and removed orp-pending labels Jul 8, 2023

cmsbuild merged commit 1ba05df into cms-sw:CMSSW_13_0_X Jul 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[13_0_X] Addition of 2023 WFs and Fixing NANO step (Fix typo) #42191

[13_0_X] Addition of 2023 WFs and Fixing NANO step (Fix typo) #42191

srimanob commented Jul 5, 2023 •

edited

Loading

cmsbuild commented Jul 5, 2023 •

edited

Loading

srimanob commented Jul 5, 2023

srimanob commented Jul 5, 2023

cmsbuild commented Jul 5, 2023

srimanob commented Jul 5, 2023 •

edited

Loading

sunilUIET commented Jul 6, 2023

cmsbuild commented Jul 6, 2023

perrotta left a comment

perrotta commented Jul 6, 2023 •

edited

Loading

cmsbuild commented Jul 6, 2023

srimanob commented Jul 6, 2023

slava77 Jul 6, 2023

slava77 Jul 6, 2023

srimanob Jul 6, 2023

sunilUIET Jul 6, 2023

cmsbuild commented Jul 6, 2023

srimanob commented Jul 7, 2023

cmsbuild commented Jul 7, 2023

srimanob commented Jul 7, 2023

sunilUIET commented Jul 8, 2023

cmsbuild commented Jul 8, 2023

perrotta commented Jul 8, 2023

[13_0_X] Addition of 2023 WFs and Fixing NANO step (Fix typo) #42191

[13_0_X] Addition of 2023 WFs and Fixing NANO step (Fix typo) #42191

Conversation

srimanob commented Jul 5, 2023 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Jul 5, 2023 • edited Loading

srimanob commented Jul 5, 2023

srimanob commented Jul 5, 2023

cmsbuild commented Jul 5, 2023

Comparison Summary

srimanob commented Jul 5, 2023 • edited Loading

sunilUIET commented Jul 6, 2023

cmsbuild commented Jul 6, 2023

perrotta left a comment

Choose a reason for hiding this comment

perrotta commented Jul 6, 2023 • edited Loading

cmsbuild commented Jul 6, 2023

srimanob commented Jul 6, 2023

slava77 Jul 6, 2023

Choose a reason for hiding this comment

slava77 Jul 6, 2023

Choose a reason for hiding this comment

srimanob Jul 6, 2023

Choose a reason for hiding this comment

sunilUIET Jul 6, 2023

Choose a reason for hiding this comment

cmsbuild commented Jul 6, 2023

RelVals-INPUT

Comparison Summary

srimanob commented Jul 7, 2023

cmsbuild commented Jul 7, 2023

Comparison Summary

srimanob commented Jul 7, 2023

sunilUIET commented Jul 8, 2023

cmsbuild commented Jul 8, 2023

perrotta commented Jul 8, 2023

srimanob commented Jul 5, 2023 •

edited

Loading

cmsbuild commented Jul 5, 2023 •

edited

Loading

srimanob commented Jul 5, 2023 •

edited

Loading

perrotta commented Jul 6, 2023 •

edited

Loading