Adding PUPPI jets and METs to offline DQM #44336

rseidita · 2024-03-07T09:04:27Z

PR description:

Adding the PUPPI collection for both jets and METs to the offline DQM JetMET sequence, as presented here.

PR validation:

echo '{
"369978" : [[1, 800]]
}' > step1_lumiRanges.log  2>&1

(dasgoclient --limit 0 --query 'lumi,file dataset=/JetMET0/Run2023D-v1/RAW run=369978' --format json | das-selected-lumis.py 1,800 ) | sort -u > step1_dasquery.log  2>&1

cmsDriver.py step2  --process reHLT -s L1REPACK:Full,HLT:@relval2023 --conditions auto:run3_hlt_relval --data  --eventcontent FEVTDEBUGHLT --datatier FEVTDEBUGHLT --era Run3_2023 -n 1000 --filein filelist:step1_dasquery.log --lumiToProcess step1_lumiRanges.log --fileout file:step2.root  --nThreads 8

cmsDriver.py step3  --conditions auto:run3_data_prompt_relval -s RAW2DIGI,L1Reco,RECO,PAT,NANO,DQM:@jetmet --datatier RECO,MINIAOD,NANOAOD,DQMIO --eventcontent RECO,MINIAOD,NANOEDMAOD,DQM --data  --process reRECO --scenario pp --era Run3_2023 --customise Configuration/DataProcessing/RecoTLR.customisePostEra_Run3 --hltProcess reHLT -n 1000 --filein  file:step2.root  --fileout file:step3.root  --nThreads 8

cmsDriver.py step4  -s HARVESTING:@jetmet --conditions auto:run3_data --data  --filetype DQM --scenario pp --era Run3_2023 -n 1000 --filein file:step3_inDQM.root --fileout file:step4.root

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport

cmsbuild · 2024-03-07T09:04:54Z

cms-bot internal usage

cmsbuild · 2024-03-07T09:13:51Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44336/39369

This PR adds an extra 64KB to repository

cmsbuild · 2024-03-07T09:14:13Z

A new Pull Request was created by @rseidita for master.

It involves the following packages:

DQMOffline/JetMET (dqm)

@tjavaid, @cmsbuild, @rvenditti, @syuvivida, @nothingface0, @antoniovagnerini can you please review it and eventually sign? Thanks.
@seemasharmafnal, @ahinzmann, @gkasieczka, @jdolen, @mariadalfonso, @mmarionncern, @rappoccio, @rociovilar, @schoef, @jdamgov, @nhanvtran this is something you requested to watch as well.
@sextonkennedy, @antoniovilela, @rappoccio you are the release manager for this.

cms-bot commands are listed here

tjavaid · 2024-03-07T09:44:22Z

please test

cmsbuild · 2024-03-07T15:55:13Z

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-35c537/37954/summary.html
COMMIT: c931ea7
CMSSW: CMSSW_14_1_X_2024-03-06-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44336/37954/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

140.56

----- Begin Fatal Exception 07-Mar-2024 15:54:46 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 326479 lumi: 7 event: 1394020 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Calling method for module ChainedJetCorrectorProducer/'dqmAk4PFPuppiL1FastL2L3ResidualCorrector'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: reco::JetCorrector
Looking for module label: ak4PFPuppiL1FastjetCorrector
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

1000.0

----- Begin Fatal Exception 07-Mar-2024 15:59:11 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 165121 lumi: 62 event: 23609118 stream: 0
   [1] Running path 'dqmoffline_3_step'
   [2] Calling method for module ChainedJetCorrectorProducer/'dqmAk4PFPuppiL1FastL2L3ResidualCorrector'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: reco::JetCorrector
Looking for module label: ak4PFPuppiL1FastjetCorrector
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

1001.0

----- Begin Fatal Exception 07-Mar-2024 16:00:13 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 165121 lumi: 62 event: 23609118 stream: 0
   [1] Running path 'dqmoffline_2_step'
   [2] Calling method for module ChainedJetCorrectorProducer/'dqmAk4PFPuppiL1FastL2L3ResidualCorrector'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: reco::JetCorrector
Looking for module label: ak4PFPuppiL1FastjetCorrector
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

138.4138.4_PromptCollisions2021/step2_PromptCollisions2021.log
138.5138.5_ExpressCollisions2021/step2_ExpressCollisions2021.log
140.61140.61_RunHI2022/step2_RunHI2022.log

Expand to see more relval errors ...

rseidita · 2024-03-08T11:02:06Z

@tjavaid I tried running the following workflows locally: 1001.4 1002.3 1002.4 1001.3 141.901 1001.2 1040.0 140.56 1000.0 1001.0 140.57 140.56 1003.0

Run successfully: 1001.4 1002.3 1002.4 1001.3 141.901 1001.2 1040.0

Fail because raw files are on tape:

140.56 RunHI2018 → On tape
140.57 RunHI2018Reduced → On tape
140.56 RunHI2018 → On tape

Fail because DAS query at step0 fails:

1000.0 RunMinBias2011A → DAS query in step0 fails
1001.0 RunMinBias2011A → DAS query in step0 fails
1003.0 RunMinBias2012A → DAS query in step0 fails

Assuming I'm testing the correct things, I don't see issues in the execution of the actual code. Please let me know if I'm looking in the wrong place

AdrianoDee · 2024-03-13T08:51:23Z

please test

cmsbuild · 2024-03-13T10:43:51Z

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-35c537/38093/summary.html
COMMIT: c931ea7
CMSSW: CMSSW_14_1_X_2024-03-12-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/44336/38093/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

140.56

----- Begin Fatal Exception 13-Mar-2024 10:17:31 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 326479 lumi: 7 event: 1394020 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Calling method for module ChainedJetCorrectorProducer/'dqmAk4PFPuppiL1FastL2L3ResidualCorrector'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: reco::JetCorrector
Looking for module label: ak4PFPuppiL1FastjetCorrector
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

1000.0

----- Begin Fatal Exception 13-Mar-2024 10:21:49 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 165121 lumi: 62 event: 23609118 stream: 0
   [1] Running path 'dqmoffline_3_step'
   [2] Calling method for module ChainedJetCorrectorProducer/'dqmAk4PFPuppiL1FastL2L3ResidualCorrector'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: reco::JetCorrector
Looking for module label: ak4PFPuppiL1FastjetCorrector
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

1001.0

----- Begin Fatal Exception 13-Mar-2024 10:22:59 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 165121 lumi: 62 event: 23609118 stream: 0
   [1] Running path 'dqmoffline_2_step'
   [2] Calling method for module ChainedJetCorrectorProducer/'dqmAk4PFPuppiL1FastL2L3ResidualCorrector'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: reco::JetCorrector
Looking for module label: ak4PFPuppiL1FastjetCorrector
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

138.4138.4_PromptCollisions2021/step2_PromptCollisions2021.log
138.5138.5_ExpressCollisions2021/step2_ExpressCollisions2021.log
140.61140.61_RunHI2022/step2_RunHI2022.log

Expand to see more relval errors ...

cmsbuild · 2024-04-17T19:39:23Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-35c537/38896/summary.html
COMMIT: 866d59b
CMSSW: CMSSW_14_1_X_2024-04-17-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44336/38896/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-35c537/38896/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-35c537/38896/git-merge-result

Comparison Summary

Summary:

You potentially added 7 lines to the logs
Reco comparison results: 2987 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3319599
DQMHistoTests: Total failures: 6039
DQMHistoTests: Total nulls: 195
DQMHistoTests: Total successes: 3313345
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 3605.085 KiB( 47 files compared)
DQMHistoSizes: changed ( 1000.0,... ): 127.810 KiB JetMET/MET
DQMHistoSizes: changed ( 1000.0 ): 43.835 KiB JetMET/Jet
DQMHistoSizes: changed ( 10224.0,... ): 43.851 KiB JetMET/Jet
DQMHistoSizes: changed ( 13234.0,... ): 0.016 KiB JetMET/Jet
Checked 202 log files, 165 edm output root files, 48 DQM output files
TriggerResults: found differences in 1 / 46 workflows

tjavaid · 2024-04-23T09:35:00Z

Hi @rseidita, @AdrianoDee , I am wondering whether changes to WF 11634.0 are expected with this update? (see for muons and L1TEMU)

rseidita · 2024-04-23T09:37:44Z

Hi @tjavaid, I can't open the links apparently, did you use the "Link me" feature of the DQM GUI?

rseidita · 2024-04-23T09:41:24Z

Regardless, this PR only touches files in the DQMOffline/JetMET package, so no it should not affect anything in other workspaces as far as I know

tjavaid · 2024-04-23T09:42:43Z

Hi @rseidita , sure. Please consider corresponding links for muons and L1TEMU workspaces.

rseidita · 2024-04-23T10:02:11Z

Thanks @tjavaid, the links work now. Looking at the logs, I see that the difference is in the HLT results for some events:

Found 10 matching events, out of which 3 have different HLT results

      Events    Accepted      Gained        Lost       Other  Trigger
          10           9          +1          -2           -  Flag_ecalLaserCorrFilter

Could this be due to a change in ECAL laser corrections? My changes are entirely restricted to the JetMET offline DQM sequence, so I don't see how they can have anything to do with trigger results and/or the muon/L1T DQM sequence. Are we sure that the tests were run on exactly matching conditions?

AdrianoDee · 2024-04-23T13:40:53Z

Differences for 11634.0 are related to the specific IB used in the tests. See #44779 (comment).

mandrenguyen · 2024-04-23T13:42:49Z

urgent
DQM signaled that the 14_0_X backport of this (#44752) is needed for the next-data taking release at joint ops yesterday.
If that's indeed the case, @cms-sw/dqm-l2 please review.

Matt (ORM)

syuvivida · 2024-04-23T15:25:50Z

+1

cmsbuild · 2024-04-23T15:26:12Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

antoniovilela · 2024-04-24T11:12:13Z

+1

Adding PUPPI jets and METs to offline DQM

c931ea7

cmsbuild added this to the CMSSW_14_1_X milestone Mar 7, 2024

cmsbuild added dqm-pending pending-signatures tests-pending orp-pending code-checks-pending labels Mar 7, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 7, 2024

cmsbuild added tests-started and removed tests-pending labels Mar 7, 2024

cmsbuild added tests-rejected and removed tests-started labels Mar 7, 2024

cmsbuild added tests-started and removed tests-rejected labels Mar 13, 2024

cmsbuild added tests-rejected and removed tests-started labels Mar 13, 2024

Fixing loading of JetCorrectors for PUPPI

1ebe746

cmsbuild added tests-pending code-checks-pending and removed tests-rejected code-checks-approved labels Mar 15, 2024

cmsbuild added requires-external tests-started and removed tests-rejected labels Apr 17, 2024

cmsbuild added tests-approved and removed tests-started labels Apr 17, 2024

makortel mentioned this pull request Apr 18, 2024

Non-reproducibility in workflow 11634.0 #44779

Open

cmsbuild added the urgent label Apr 23, 2024

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Apr 23, 2024

cmsbuild added orp-approved and removed orp-pending labels Apr 24, 2024

cmsbuild merged commit 83deecd into cms-sw:master Apr 24, 2024
11 checks passed

rseidita deleted the jme-offline-dqm-puppi branch April 24, 2024 12:09

This was referenced Apr 24, 2024

Add script to merge the JSON files produced by the FastTimerService #44836

Merged

Phase2 OMTF emulator #44557

Merged

Adaptions for Pythia 8.311 #43786

Merged

[GCC13] Fix Woverloaded-virtual warning #44581

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding PUPPI jets and METs to offline DQM #44336

Adding PUPPI jets and METs to offline DQM #44336

rseidita commented Mar 7, 2024

cmsbuild commented Mar 7, 2024 •

edited

Loading

cmsbuild commented Mar 7, 2024

cmsbuild commented Mar 7, 2024

tjavaid commented Mar 7, 2024

cmsbuild commented Mar 7, 2024

rseidita commented Mar 8, 2024

AdrianoDee commented Mar 13, 2024

cmsbuild commented Mar 13, 2024

cmsbuild commented Apr 17, 2024

tjavaid commented Apr 23, 2024

rseidita commented Apr 23, 2024

rseidita commented Apr 23, 2024

tjavaid commented Apr 23, 2024

rseidita commented Apr 23, 2024

AdrianoDee commented Apr 23, 2024 •

edited

Loading

mandrenguyen commented Apr 23, 2024

syuvivida commented Apr 23, 2024

cmsbuild commented Apr 23, 2024

antoniovilela commented Apr 24, 2024

Adding PUPPI jets and METs to offline DQM #44336

Adding PUPPI jets and METs to offline DQM #44336

Conversation

rseidita commented Mar 7, 2024

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Mar 7, 2024 • edited Loading

cmsbuild commented Mar 7, 2024

cmsbuild commented Mar 7, 2024

tjavaid commented Mar 7, 2024

cmsbuild commented Mar 7, 2024

RelVals

RelVals-INPUT

rseidita commented Mar 8, 2024

AdrianoDee commented Mar 13, 2024

cmsbuild commented Mar 13, 2024

RelVals

RelVals-INPUT

cmsbuild commented Apr 17, 2024

Comparison Summary

tjavaid commented Apr 23, 2024

rseidita commented Apr 23, 2024

rseidita commented Apr 23, 2024

tjavaid commented Apr 23, 2024

rseidita commented Apr 23, 2024

AdrianoDee commented Apr 23, 2024 • edited Loading

mandrenguyen commented Apr 23, 2024

syuvivida commented Apr 23, 2024

cmsbuild commented Apr 23, 2024

antoniovilela commented Apr 24, 2024

cmsbuild commented Mar 7, 2024 •

edited

Loading

AdrianoDee commented Apr 23, 2024 •

edited

Loading