Online DQM: add an input argument outputBaseDir in pbsource_cfi.py (to fit with the new hltd version)

[syuvivida]

PR description:

This is a follow-up and update with respect to PR 41986.

After testing the hlt clients in the new online DQM machines [0][1], we find an update in the DQM/Integration/python/config/pbsource_cfi.py is also required when using the new hltd version in the new online DQM machines: an additional input argument outputBaseDir must be provided.

For the current (old) online DQM machines, whether adding this input argument or not has no effect.

More information about the upgrade of new DQM machines below:
We are working on the upgrade of online DQM machines [0][1]. There will be a few months that we share the same CMSSW code between the new and the current machines.
In the current (old) DQM machines, the disks of bu-c2f11-09-01 and bu-c2f11-13-01 are mounted on our fu machines as /fff/BU0. Event display clients visualization-live and visualization-live-secondInstance produce output root files at /fff/BU0/output.

However, the mount point (path) has changed in the new online DQM machines [1]. In order to use the same event display client codes for both old and new machines and also to make the path name more flexible, we replace the output path with an input argument (with a default value of /fff/BU0/output). The old machines will use an old hltd version and take the default value of the argument outputBaseDir, while the input values for the new machines will be determined by hltd and startDqmRun.sh.

[0] twiki about the upgrade of DQM machines
[1] JIRA ticket that includes the communication with DAQ
[2] JIRA ticket of the tests during TS1

PR validation:

• This PR has been tested at lxplus by running the hlt, hcal, and ecal clients standalone with CMSSW_13_0_X_2023-06-27-1100 , CMSSW_13_1_X_2023-06-27-1100, and CMSSW_13_2_X_2023-06-26-2300 with the streamers at /eos/cms/store/group/comm_dqm/Collisions23_tempStreamers/.
• This PR has been tested at the current (old) online DQM playback machines and ran all clients without problem when using the default value of the input argument.
• This PR has been deployed/tested when we tested the data transfer during TS1 [2].

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42128/36107

• This PR adds an extra 12KB to repository

[cmsbuild]

A new Pull Request was created by @syuvivida for master.

It involves the following packages:

• DQM/Integration (dqm)

@nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@threus, @batinkov, @francescobrivio this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[emanueleusai]

please test

[cmsbuild]

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-85feeb/33507/summary.html
COMMIT: 5cedebe
CMSSW: CMSSW_13_2_X_2023-07-02-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42128/33507/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

• 10224.0

----- Begin Fatal Exception 03-Jul-2023 07:21:46 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root
      [c] Input file root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.�
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/17fac9a9-98f1-43d3-9dbd-d26d638e04dd.root?tried=+1213xrootd-cms-redir-int.cr.cnaf.infn.it,&xrdcl.requuid=13da4702-0940-4900-a700-2a7b010a0c1d
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

• 23.023.0_JpsiMM/step2_JpsiMM.log
• 4.294.29_RunMinBias2011B/step2_RunMinBias2011B.log
• 136.741136.741_RunMuonEG2016C/step2_RunMuonEG2016C.log

Expand to see more relval errors ...

• 134.808
• 134.806
• 134.907
• 136.754
• 136.752
• 136.756
• 136.75
• 136.749
• 136.795
• 136.803
• 136.832
• 136.839
• 136.869
• 1325.517
• 1347.0
• 13992502.0
• 134.99601
• 134.0
• 10007.0
• 10004.0
• 10803.0
• 250206.18

[emanueleusai]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-85feeb/33553/summary.html
COMMIT: 5cedebe
CMSSW: CMSSW_13_2_X_2023-07-04-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42128/33553/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 2 lines from the logs
• Reco comparison results: 2 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3193832
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3193807
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 207 log files, 159 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[emanueleusai]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1