Fix HLT customization for alpaka PF #43999

waredjeb · 2024-02-16T16:20:50Z

This PR fixes the customization of Alpaka-PF introduced with #43971 with moving the CPU serial module to DQM_HcalReconstruction_v6

GPU vs CPU comparison at DQMOnline will followup, currently we are only saving the needed collections

Validated running:

https_proxy=http://cmsproxy.cms:3128/ hltGetConfiguration /dev/CMSSW_14_0_0/GRun --unprescale --output all --globaltag auto:phase1_2024_realistic --mc --max-events 10 --input /store/mc/Run3Winter24Digi/TT_TuneCP5_13p6TeV_powheg-pythia8/GEN-SIM-RAW/133X_mcRun3_2024_realistic_v8-v2/80000/dc984f7f-2e54-48c4-8950-5daa848b6db9.root --customise HLTrigger/Configuration/customizeHLTforAlpaka.customizeHLTforAlpaka > hlt.py

Backport:
#44000

cmsbuild · 2024-02-16T16:21:14Z

cms-bot internal usage

fwyzard · 2024-02-16T16:23:34Z

type bugfix

cmsbuild · 2024-02-16T16:28:54Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43999/38919

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File HLTrigger/Configuration/python/customizeHLTforAlpaka.py modified in PR(s): Alpaka vs CUDA DQM compare modules for pixel tracks objects #43964

cmsbuild · 2024-02-16T16:29:14Z

A new Pull Request was created by @waredjeb for master.

It involves the following packages:

HLTrigger/Configuration (hlt)

@cmsbuild, @Martin-Grunewald, @mmusich can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @silviodonato this is something you requested to watch as well.
@rappoccio, @sextonkennedy, @antoniovilela you are the release manager for this.

cms-bot commands are listed here

fwyzard · 2024-02-16T16:31:50Z

please test

Martin-Grunewald · 2024-02-16T16:32:34Z

HLTrigger/Configuration/python/customizeHLTforAlpaka.py

@@ -249,6 +243,21 @@ def replaceItemsInSequence(process, itemsToReplace, replacingSequence):
    itemsList = [process.hltParticleFlowRecHitHBHE, process.hltParticleFlowClusterHBHE,process.hltParticleFlowClusterHCAL]
    process = replaceItemsInSequence(process, itemsList, process.HLTPFHcalClustering)

+    process.PFClusterHBHECPUSerial = cms.Sequence(process.hltHBHERecHitToSoA+process.hltPFRecHitSoAProducerHCALCPUSerial+process.hltPFClusterSoAProducerCPUSerial)


HLT sequences start with uppercase HLT so rename to HLTPFClusterHBHECPUSerial and propagate.

Where is this sequence used anyway?

The Sequence is then added to DQM_HcalReconstruction_v6

fwyzard · 2024-02-16T16:33:09Z

Looks good to me.

I suggested to @waredjeb to reuse the HCAL DQM path instead of creating a new one to keep the customisation simpler.

Martin-Grunewald · 2024-02-16T16:35:04Z

HLTrigger/Configuration/python/customizeHLTforAlpaka.py

+    dqmHcalRecoPathName = "DQM_HcalReconstruction_v6"
+    dqmHcalPath= getattr(process, dqmHcalRecoPathName)
+    dqmHcalRecoPathIndex = dqmHcalPath.index(process.hltHcalConsumerGPU) + 1
+    dqmHcalPath.insert(dqmHcalRecoPathIndex , process.PFClusterHBHECPUSerial)


Ah ok, here - please update.

fwyzard · 2024-02-16T16:41:53Z

please test

cmsbuild · 2024-02-16T16:46:52Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43999/38920

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File HLTrigger/Configuration/python/customizeHLTforAlpaka.py modified in PR(s): Alpaka vs CUDA DQM compare modules for pixel tracks objects #43964

cmsbuild · 2024-02-16T16:47:12Z

Pull request #43999 was updated. @Martin-Grunewald, @mmusich can you please check and sign again.

cmsbuild · 2024-02-16T19:21:19Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-703286/37532/summary.html
COMMIT: 0a56692
CMSSW: CMSSW_14_1_X_2024-02-15-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43999/37532/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 98 lines from the logs
Reco comparison results: 39 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3248554
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3248529
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 200 log files, 161 edm output root files, 48 DQM output files
TriggerResults: no differences found

Martin-Grunewald · 2024-02-17T06:39:27Z

+1

cmsbuild · 2024-02-17T06:39:46Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @sextonkennedy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

waredjeb · 2024-02-19T09:16:26Z

preparing a fix addressing comment

waredjeb · 2024-02-19T10:08:56Z

Ok since the fix would touch also the RECO I might open a different pull request and this one can get merged if you all agree.

Martin-Grunewald · 2024-02-19T12:17:04Z

OK with me! (again, for 14_1 and 14_0).

antoniovilela · 2024-02-19T23:52:41Z

+1

makortel · 2024-02-21T17:27:34Z

HLTrigger/Configuration/python/customizeHLTforAlpaka.py

+
+    # Add CPUSerial sequences to DQM_HcalReconstruction_v6 Path
+    dqmHcalRecoPathName = "DQM_HcalReconstruction_v6"
+    dqmHcalPath= getattr(process, dqmHcalRecoPathName)


Seems like this getattr() call is causing workflow failures in GPU IB, see #44046

I guess something like this

cmssw/HLTrigger/Configuration/python/customizeHLTforAlpaka.py

Lines 268 to 275 in 8f7db2f

dqmPixelRecoPathName = None

for pathName in process.paths_():

if pathName.startswith('DQM_PixelReconstruction_v'):

dqmPixelRecoPathName = pathName

break

if dqmPixelRecoPathName == None:

return process

is needed.
@waredjeb FYI

thanks for providing the fix with #44048
should I update #44000 with the same fix?

should I update #44000 with the same fix?

yes, please

mmusich · 2024-02-21T18:11:24Z

type pf

cmsbuild added this to the CMSSW_14_1_X milestone Feb 16, 2024

cmsbuild added hlt-pending pending-signatures tests-pending orp-pending code-checks-pending labels Feb 16, 2024

waredjeb mentioned this pull request Feb 16, 2024

[14.0.X] Fix HLT customization for alpaka PF #44000

Merged

cmsbuild added the bug-fix label Feb 16, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 16, 2024

cmsbuild added tests-started and removed tests-pending labels Feb 16, 2024

Martin-Grunewald reviewed Feb 16, 2024

View reviewed changes

remove CPU serial and add it to DQM_HcalReconstruction_v6

0a56692

waredjeb force-pushed the fix_integratePFAlpakaClusteringAtHLT_14_0_0_pre3 branch from 0418195 to 0a56692 Compare February 16, 2024 16:38

cmsbuild added tests-pending code-checks-pending and removed tests-started code-checks-approved labels Feb 16, 2024

cmsbuild added tests-started and removed tests-pending labels Feb 16, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 16, 2024

cmsbuild added tests-approved and removed tests-started labels Feb 16, 2024

cmsbuild added hlt-approved fully-signed and removed hlt-pending pending-signatures labels Feb 17, 2024

cmsbuild added orp-approved and removed orp-pending labels Feb 19, 2024

cmsbuild merged commit c1bc0fc into cms-sw:master Feb 19, 2024
21 checks passed

This was referenced Feb 20, 2024

Updated root to tip of branch master cms-sw/cmsdist#9025

Closed

Updated root to tip of branch v6-28-00-patches cms-sw/cmsdist#9027

Merged

makortel mentioned this pull request Feb 21, 2024

[GPU] Workflow failures from missing DQM_HcalReconstruction_v6 #44046

Closed

makortel reviewed Feb 21, 2024

View reviewed changes

mmusich mentioned this pull request Feb 21, 2024

Fix customizeHLTforAlpakaParticleFlowClustering: insert DQM_HcalReconstruction_v only if present in the HLT menu #44048

Merged

cmsbuild added the pf label Feb 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix HLT customization for alpaka PF #43999

Fix HLT customization for alpaka PF #43999

waredjeb commented Feb 16, 2024 •

edited

Loading

cmsbuild commented Feb 16, 2024 •

edited

Loading

fwyzard commented Feb 16, 2024

cmsbuild commented Feb 16, 2024

cmsbuild commented Feb 16, 2024

fwyzard commented Feb 16, 2024

Martin-Grunewald Feb 16, 2024

fwyzard Feb 16, 2024

Martin-Grunewald Feb 16, 2024

waredjeb Feb 16, 2024

fwyzard commented Feb 16, 2024

Martin-Grunewald Feb 16, 2024

fwyzard commented Feb 16, 2024

cmsbuild commented Feb 16, 2024

cmsbuild commented Feb 16, 2024

cmsbuild commented Feb 16, 2024

Martin-Grunewald commented Feb 17, 2024

cmsbuild commented Feb 17, 2024

waredjeb commented Feb 19, 2024

waredjeb commented Feb 19, 2024

Martin-Grunewald commented Feb 19, 2024

antoniovilela commented Feb 19, 2024

makortel Feb 21, 2024

mmusich Feb 21, 2024

waredjeb Feb 21, 2024

mmusich Feb 21, 2024

mmusich commented Feb 21, 2024

	dqmPixelRecoPathName = None
	for pathName in process.paths_():
	if pathName.startswith('DQM_PixelReconstruction_v'):
	dqmPixelRecoPathName = pathName
	break

	if dqmPixelRecoPathName == None:
	return process

Fix HLT customization for alpaka PF #43999

Fix HLT customization for alpaka PF #43999

Conversation

waredjeb commented Feb 16, 2024 • edited Loading

cmsbuild commented Feb 16, 2024 • edited Loading

fwyzard commented Feb 16, 2024

cmsbuild commented Feb 16, 2024

cmsbuild commented Feb 16, 2024

fwyzard commented Feb 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fwyzard commented Feb 16, 2024

Choose a reason for hiding this comment

fwyzard commented Feb 16, 2024

cmsbuild commented Feb 16, 2024

cmsbuild commented Feb 16, 2024

cmsbuild commented Feb 16, 2024

Comparison Summary

Martin-Grunewald commented Feb 17, 2024

cmsbuild commented Feb 17, 2024

waredjeb commented Feb 19, 2024

waredjeb commented Feb 19, 2024

Martin-Grunewald commented Feb 19, 2024

antoniovilela commented Feb 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mmusich commented Feb 21, 2024

waredjeb commented Feb 16, 2024 •

edited

Loading

cmsbuild commented Feb 16, 2024 •

edited

Loading