JME DQM implementation of Z/Gamma plus Jet monitoring for Run3 (14_1_X) #44411
Conversation
|
cms-bot internal usage |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44411/39486
|
|
A new Pull Request was created by @esiam for master. It involves the following packages:
@syuvivida, @tjavaid, @rvenditti, @cmsbuild, @antoniovagnerini, @nothingface0 can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
|
please test |
| unsigned index = triggerNames_.triggerIndex(pathName); // find in which index is that path | ||
| if (!(hltcfgIndex == index)) { | ||
| if (print) | ||
| std::cout << "Error in trigger index" << std::endl; |
There was a problem hiding this comment.
no cout in production code, please use MessageLogger.
| if (index > 0 && index < triggerNames_.size() && | ||
| triggerResults->accept(index)) { //trigger is accepted and index is valid | ||
| if (print) | ||
| std::cout << "Trigger accepted" << std::endl; |
There was a problem hiding this comment.
no cout in production code, please use MessageLogger.
| std::vector<std::string> module_names = hltConfig_.moduleLabels(index); // (pathname) works too | ||
| if (!(module_size == module_names.size())) { | ||
| if (print) | ||
| std::cout << "ERROR IN MODULES COUNTING" << std::endl; |
There was a problem hiding this comment.
no cout in production code, please use MessageLogger.
| } | ||
| if (module_size == 0) { | ||
| if (print) | ||
| std::cout << "no modules in this path ?!?!" << std::endl; |
There was a problem hiding this comment.
no cout in production code, please use MessageLogger.
| edm::InputTag moduleFilter; | ||
| moduleFilter = edm::InputTag(moduleName, "", processName_); | ||
| if (print) | ||
| std::cout << moduleFilter << std::endl; |
There was a problem hiding this comment.
no cout in production code, please use MessageLogger.
| const bool doDirectBalancevsReferencePt = true, | ||
| const bool bookDen = false); | ||
|
|
||
| void FillME(ObjME* a_me, |
There was a problem hiding this comment.
coding rule 2.8: from https://cms-sw.github.io/cms_coding_rules.html:
Start method names with lowercase, use upper case initials for following words,
| edm::EDGetTokenT<trigger::TriggerEvent> triggerEventObject_; | ||
| edm::EDGetTokenT<edm::TriggerResults> triggerResultsToken_; |
There was a problem hiding this comment.
| edm::EDGetTokenT<trigger::TriggerEvent> triggerEventObject_; | |
| edm::EDGetTokenT<edm::TriggerResults> triggerResultsToken_; | |
| const edm::EDGetTokenT<trigger::TriggerEvent> triggerEventObject_; | |
| const edm::EDGetTokenT<edm::TriggerResults> triggerResultsToken_; |
as they are in the initializer list.
| const std::string pathName; | ||
| const std::string moduleName; | ||
| edm::InputTag jetInputTag_; | ||
| edm::EDGetTokenT<reco::PFJetCollection> jetToken_; |
There was a problem hiding this comment.
| edm::EDGetTokenT<reco::PFJetCollection> jetToken_; | |
| const edm::EDGetTokenT<reco::PFJetCollection> jetToken_; |
as it is in the initializer list.
| edm::EDGetTokenT<edm::TriggerResults> triggerResultsToken_; | ||
| const std::string pathName; | ||
| const std::string moduleName; | ||
| edm::InputTag jetInputTag_; |
There was a problem hiding this comment.
| edm::InputTag jetInputTag_; |
unused.
| } | ||
| if (!passTrig) { | ||
| if (print) | ||
| std::cout << "Trigger did not pass" << std::endl; // skip event if trigger fails |
There was a problem hiding this comment.
no cout in production code, please use MessageLogger.
aren't you targeting 2024 pp data-taking? if yes, a backport will be needed. |
| from JetMETCorrections.Configuration.JetCorrectors_cff import * | ||
|
|
||
| ak4PFJetsPuppiCorrected = cms.EDProducer('CorrectedPFJetProducer', | ||
| src = cms.InputTag('ak4PFJetsPuppi'), |
There was a problem hiding this comment.
Hi! Thanks for the comments! I will change them. Yes, it targets 2024 pp data-taking, sorry I was not sure for the backport.
There was a problem hiding this comment.
Hi! I want to ask since this happened because Puppi Jets were not available in the input sample, but if the producer were not in cff it would not give an error. This is something we should avoid in general when using corrections in DQM? What I did is use corrections on the fly. I rerun and took same distributions and also do the code checks.
Thanks, Eirini.
There was a problem hiding this comment.
I rerun and took same distributions and also do the code checks.
thank you for the updates.
I want to ask since this happened because Puppi Jets were not available in the input sample, but if the producer were not in cff it would not give an error. This is something we should avoid in general when using corrections in DQM?
I know very little of the underlying motivation behind this PR, so I don't feel qualified to comment if you should "in general" avoid these corrections.
Allow me to point out the following facts:
- all the other test workflows actually succeeded: JME DQM implementation of Z/Gamma plus Jet monitoring for Run3 (14_1_X) #44411 (comment)
- the only 2 workflow with problems 136.7801 and 136.7803 are starting from AOD and not from RAW data [1]
In both workflows the DQM sequence run is DQM:offlineHLTSourceOnAOD which is defined here
cmssw/DQMOffline/Trigger/python/DQMOffline_Trigger_cff.py
Lines 124 to 146 in 8731f56
my naive intuition would be that in case you want to run the corrector for the worfklows starting from RAW, you can modify the definition of offlineHLTSourceOnAOD such that instead of running the sequence jetmetMonitorHLT runs a different sequence (same as the regular one, but without the crashing producer, or a modified producer that could work on AOD).
Please let me know if that makes sense to you.
[1]
$ runTheMatrix.py -nel 136.7801,136.7803
136.7801 RunHLTPhy2017B_AOD+DQMHLTonAOD_2017+HARVESTDQMHLTonAOD_2017 [1]: input from: /HLTPhysics/Run2017B-PromptReco-v1/AOD with run []
[2]: cmsDriver.py step2 -s DQM:offlineHLTSourceOnAOD --conditions auto:run2_data --data --eventcontent DQM --datatier DQMIO --era Run2_2017 --fileout DQMHLTonAOD.root --procModifiers tau_readOldDiscriminatorFormat -n 100
[3]: cmsDriver.py step3 -s HARVESTING:hltOfflineDQMClient --conditions auto:run2_data --data --filetype DQM --scenario pp --era Run2_2017 --filein file:DQMHLTonAOD.root -n 100
136.7803 RunHLTPhy2017B_RAWAOD+DQMHLTonRAWAOD_2017+HARVESTDQMHLTonAOD_2017 [1]: input from: /HLTPhysics/Run2017B-PromptReco-v1/AOD with run []
[2]: cmsDriver.py step2 --process reHLT -s HLT:@relval2017,DQM:offlineHLTSourceOnAOD --conditions auto:run2_data --data --eventcontent DQM --datatier DQMIO --era Run2_2017 --secondfilein filelist:step1_dasparentquery.log --customise_commands "process.HLTAnalyzerEndpath.remove(process.hltL1TGlobalSummary)" --fileout DQMHLTonAOD.root --procModifiers tau_readOldDiscriminatorFormat -n 100
[3]: cmsDriver.py step3 -s HARVESTING:hltOfflineDQMClient --conditions auto:run2_data --data --filetype DQM --scenario pp --era Run2_2017 --filein file:DQMHLTonAOD.root -n 100
2 workflows with 3 steps
There was a problem hiding this comment.
Hi! and thanks for the reply!!
To me (I mean how I understand this fail) the problem is that the ak4PFJetsPuppi collection is not there. The exception in the log file is the following:
----- Begin Fatal Exception 15-Mar-2024 11:53:55 CET-----------------------
An exception of category 'ProductNotFound' occurred while
[0] Processing Event run: 297227 lumi: 1 event: 53500 stream: 0
[1] Running path 'dqmoffline_step'
[2] Calling method for module CorrectedPFJetProducer/'ak4PFJetsPuppiCorrected'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vectorreco::PFJet
Looking for module label: ak4PFJetsPuppi
Looking for productInstanceName:
Additional Info:
[a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------
Because of the producer. But if for example I use another (not puppi) Jet collection this would not complain. So in order to be sure that in case that our collection is missing, it would not give a fatal exception we should not use the producer and use another way to apply corrections or modify it. But we (JME) want to use Puppi for offline monitoring so, I want to ask because I do not know are they not included in AOD files or this fail is because this is an older workflow and they (Puppi Jets) were not included then?
Thanks, Eirini.
There was a problem hiding this comment.
I want to ask because I do not know are they not included in AOD files or this fail is because this is an older workflow and they (Puppi Jets) were not included then?
I think this question should be answered by @cms-sw/jetmet-pog-l2 . I see that the failing workflows are from Run2 (2017). If back then puppi jets were not produced it would be understood why the collection is not available in old AOD data . Let me ask again if you really want this to run this on workflows starting from AOD. If yes, I guess you could use an era modifier to remove the producer in the case of 2017 (or other years), otherwise the suggestion at #44411 (comment) still holds.
There was a problem hiding this comment.
Hi and thanks for your reply. In my opinion, I think we should keep the sequence there. By applying the offline corrections to be calculated on the fly (after the latest push)
- runTheMatrix.py -l limited -i all --ibeos and the two that had the fatal exception
- runTheMatrix.py -l 136.7803 --ibeos
- runTheMatrix.py -l 136.7801 --ibeos are passing. So, for now I think it is ok.
For the second question I asked, I was trying to understand why the offline Puppi collections were missing and if this is because of the file type AOD.
Thanks a lot for your commends and time,
Eirini.
There was a problem hiding this comment.
In my opinion, I think we should keep the sequence there.
removing the sequence was never an option. The suggestion was to modify it such that it can run also on old 2017 AOD data.
There was a problem hiding this comment.
Hello @mmusich , thanks for all the comments.
I think there is a difference in what each calls sequence above, and just to make sure we have converged on this point I am summarizing the situation :
What Rena was referring above by "removing the sequence" is "removing the corrections sequence" of ak4PFPuppiL1FastL2L3CorrectorChain from here.
So in this sense after her latest push this is not needed to run. The reason is because the "correctors" part will always work even if the ak4PFJetsPuppi does not exist. It only needs the AK4PFPuppi tag in the GT (but that should anyway exist in any GT running in 14_X otherwise all correctors that use it in CMSSW would crash).
Now then in the latest push that she did, the need to have ak4PFJetsPuppi is treated by not producing the corrected jets though the "CorrectedJetsProducer" which would need the collection to work (so in practice modified this sequence for DQM to work in 2017 - Marco's suggestion). The correction is applied then inside the .cc part of the DQM code. If the collection does not exist in this code, like in 2017 AOD files, DQM will just return with no crash.
|
|
||
| muon_1 = v1; | ||
| muon_2 = v2; | ||
| bool muon_pass = muon_1.Pt() > muon_pt && muon_2.Pt() > muon_pt && abs(muon_1.Eta()) < muon_eta && |
There was a problem hiding this comment.
| bool muon_pass = muon_1.Pt() > muon_pt && muon_2.Pt() > muon_pt && abs(muon_1.Eta()) < muon_eta && | |
| bool muon_pass = muon_1.Pt() > muon_pt && muon_2.Pt() > muon_pt && std::abs(muon_1.Eta()) < muon_eta && |
| muon_1 = v1; | ||
| muon_2 = v2; | ||
| bool muon_pass = muon_1.Pt() > muon_pt && muon_2.Pt() > muon_pt && abs(muon_1.Eta()) < muon_eta && | ||
| abs(muon_2.Eta()) < muon_eta; |
There was a problem hiding this comment.
| abs(muon_2.Eta()) < muon_eta; | |
| std::abs(muon_2.Eta()) < muon_eta; |
| ObjME a_ME_HE_I_mediumRefPt[6]; | ||
| ObjME a_ME_HE_O_mediumRefPt[6]; | ||
| ObjME a_ME_HF_mediumRefPt[6]; | ||
| ; |
|
-1 Failed Tests: RelVals-INPUT RelVals-INPUT
Comparison SummarySummary:
|
esiam
force-pushed
the
JME_ZGammaplusJetMonitor
branch
from
7aaebe9
to
522799f
Compare
cmsbuild
added
tests-pending
code-checks-pending
and removed
tests-rejected
code-checks-approved
labels
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44411/39508
|
|
Pull request #44411 was updated. @cmsbuild, @nothingface0, @syuvivida, @rvenditti, @antoniovagnerini, @tjavaid can you please check and sign again. |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44411/39712
|
|
Pull request #44411 was updated. @nothingface0, @rvenditti, @cmsbuild, @antoniovagnerini, @tjavaid, @syuvivida can you please check and sign again. |
|
@cmsbuild, please test |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a9e10a/38473/summary.html Comparison SummarySummary:
|
|
@cms-sw/dqm-l2 can you please review this? It would be good to have this integrated once we deliver the next HLT menu 2024 V1.1 |
|
+1 |
cmsbuild
added
dqm-approved
fully-signed
and removed
dqm-pending
pending-signatures
labels
|
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @sextonkennedy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2) |
|
+1 |
|
@esiam will you prepare a 14.0.x backport? |
|
Hi Marco, yes I will. Thank you! |
PR description:
Adding DQM code for Jets monitoring with Z/Gamma plus Jets analysis module. These changes were discussed with JME trigger group. @theochatzis , @slehti
You can find a first presentation for the motivation in TSG ( @missirol , Silvia Goy Lopez ) group meeting here
Besides, you can see some plots in this link These plots were produced with 14_0 CMSSW version by rerunning the hlt for a muon dataset for 60k events.
PR validation:
The implementations were tested by running DQM steps and see that the histos are produced. Besides they were tested by using the RunTheMatrix.py -l 141.044.
If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
This is not a backport. It may be backported in CMSSW_14_0_X if needed.