Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history
commit 35a0252
Author: Leszek Grzanka <leszek.grzanka@cern.ch>
Date:   Sat Feb 24 18:27:48 2024 +0100

    silencing logs in the relvals; unit test refreshed

commit 5bef250
Author: Leszek Grzanka <grzanka@agh.edu.pl>
Date:   Sat Feb 24 15:42:54 2024 +0100

    Update Calibration/PPSAlCaRecoProducer/test/test_prompt_AlCaRecoProducer.sh

    Co-authored-by: Marco Musich <marco.musich@cern.ch>

commit 58b9a76
Author: Leszek Grzanka <grzanka@agh.edu.pl>
Date:   Sat Feb 24 14:25:30 2024 +0100

    Update Calibration/PPSAlCaRecoProducer/test/test_express_AlCaRecoProducer.sh

    Co-authored-by: Marco Musich <marco.musich@cern.ch>

commit 671fafa
Author: Leszek Grzanka <leszek.grzanka@cern.ch>
Date:   Thu Feb 22 21:26:37 2024 +0100

    renaming gtStage2Digis according to PR review; unit test refreshed

commit b031e8e
Merge: ca93802 e15c1e3
Author: Leszek Grzanka <leszek.grzanka@cern.ch>
Date:   Tue Feb 20 15:31:37 2024 +0100

    Merged CMSSW_14_1_X_feature_hltGtStage2Digis_in_ALCARECO from repository CTPPS with cms-merge-topic

commit e15c1e3
Author: abellora <andrea.bellora@cern.ch>
Date:   Thu Feb 15 15:02:18 2024 +0100

    Added hltGtStage2Digis to seqALCARECOPPSCalMaxTracksReco

Co-authored-by: Leszek Grzanka <grzanka@agh.edu.pl>
Co-authored-by: Marco Musich <marco.musich@cern.ch>
Co-authored-by: abellora <andrea.bellora@cern.ch>
  • Loading branch information
4 people committed Feb 26, 2024
1 parent ca93802 commit 6e62922
Show file tree
Hide file tree
Showing 10 changed files with 131 additions and 72 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
'keep recoForwardProtons_ctppsProtonsAlCaRecoProducer_*_*',

# HLT info
'keep *_hltGtStage2ObjectMap_*_*',
'keep *_gtStage2DigisAlCaRecoProducer_*_*',
'keep edmTriggerResults_*_*_*',
'keep triggerTriggerEvent_*_*_*'
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,18 @@
from EventFilter.CTPPSRawToDigi.ctppsRawToDigi_cff import ctppsDiamondRawToDigi as _ctppsDiamondRawToDigi
from EventFilter.CTPPSRawToDigi.ctppsRawToDigi_cff import totemTimingRawToDigi as _totemTimingRawToDigi
from EventFilter.CTPPSRawToDigi.ctppsRawToDigi_cff import ctppsPixelDigis as _ctppsPixelDigis
from EventFilter.L1TRawToDigi.gtStage2Digis_cfi import gtStage2Digis as _gtStage2Digis

ctppsDiamondRawToDigiAlCaRecoProducer = _ctppsDiamondRawToDigi.clone(rawDataTag = 'hltPPSCalibrationRaw')
totemTimingRawToDigiAlCaRecoProducer = _totemTimingRawToDigi.clone(rawDataTag = 'hltPPSCalibrationRaw')
ctppsPixelDigisAlCaRecoProducer = _ctppsPixelDigis.clone(inputLabel = 'hltPPSCalibrationRaw')
gtStage2DigisAlCaRecoProducer = _gtStage2Digis.clone(InputLabel="hltFEDSelectorL1")

ctppsRawToDigiTaskAlCaRecoProducer = cms.Task(
ctppsDiamondRawToDigiAlCaRecoProducer,
totemTimingRawToDigiAlCaRecoProducer,
ctppsPixelDigisAlCaRecoProducer
ctppsPixelDigisAlCaRecoProducer,
gtStage2DigisAlCaRecoProducer
)

ALCARECOPPSCalMaxTracksRaw2Digi = cms.Sequence(ctppsRawToDigiTaskAlCaRecoProducer)
Expand Down Expand Up @@ -102,4 +105,4 @@
# 6. master sequence object
#------------------------------------------------------

seqALCARECOPPSCalMaxTracksReco = cms.Sequence( ALCARECOPPSCalMaxTracksFilter + ALCARECOPPSCalMaxTracksRaw2Digi + recoPPSSequenceAlCaRecoProducer)
seqALCARECOPPSCalMaxTracksReco = cms.Sequence( ALCARECOPPSCalMaxTracksFilter + ALCARECOPPSCalMaxTracksRaw2Digi + recoPPSSequenceAlCaRecoProducer)
26 changes: 26 additions & 0 deletions Calibration/PPSAlCaRecoProducer/test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Running tests on lxplus


## Using scram

To be able to run tests locally enable re-director used internally by `cmsbuild`:

```
CMS_PATH="/cvmfs/cms-ib.cern.ch" SITECONFIG_PATH="/cvmfs/cms-ib.cern.ch/SITECONF/local" scram b runtests
```

## Manually (to inspect test output files)

To run the tests in this directory (`Calibration/PPSAlCaRecoProducer/test/`) type:

```
CMS_PATH="/cvmfs/cms-ib.cern.ch" SITECONFIG_PATH="/cvmfs/cms-ib.cern.ch/SITECONF/local" ./test_express_AlCaRecoProducer.sh
```

and

```
SCRAM_TEST_PATH=. ./test_express_PPSAlCaReco_output.sh
```

The same can be done for prompt tests
Original file line number Diff line number Diff line change
@@ -1,29 +1,32 @@
#!/bin/bash
function die { echo $1: status $2; exit $2; }

customise_commands="process.GlobalTag.toGet = cms.VPSet()\
\nprocess.GlobalTag.toGet.append(cms.PSet(record = cms.string(\"AlCaRecoTriggerBitsRcd\"),tag = cms.string(\"AlCaRecoHLTpaths_PPS2022_express_v1\"), connect = cms.string(\"frontier://FrontierProd/CMS_CONDITIONS\")))\
\nprocess.GlobalTag.toGet.append(cms.PSet(record = cms.string(\"PPSTimingCalibrationLUTRcd\"),tag = cms.string(\"PPSDiamondTimingCalibrationLUT_test\"), connect = cms.string(\"frontier://FrontierProd/CMS_CONDITIONS\")))\
\nprocess.ALCARECOPPSCalMaxTracksFilter.TriggerResultsTag = cms.InputTag(\"TriggerResults\",\"\",\"HLTX\")"
# customisation command needed for all tests cases
COMMON_CUSTOM="process.ALCARECOPPSCalMaxTracksFilter.TriggerResultsTag = cms.InputTag(\"TriggerResults\",\"\",\"HLTX\")"

INPUTFILE="/store/group/alca_global/pps_alcareco_producer_tests/outputALCAPPS_single.root"
COMMMAND=`xrdfs cms-xrd-global.cern.ch locate $INPUTFILE`
STATUS=$?
echo "xrdfs command status = "$STATUS
if [ $STATUS -eq 0 ]; then
echo "Using file ${INPUTFILE}. Running in ${SCRAM_TEST_PATH}."
# test on 2022 data
INPUTFILE_355207="/store/data/Run2022B/AlCaPPS/RAW/v1/000/355/207/00000/c23440f4-49c0-44aa-b8f6-f40598fb4705.root"

# new test on generated data, with same structure as data expected in 2024
INPUTFILE_367104="/store/group/alca_global/pps_alcareco_producer_tests/outputALCAPPSExpress.root"

# all input files and customisation commands to loop through
INPUTFILES=($INPUTFILE_355207 $INPUTFILE_367104)

# test case loop
for TEST_RUN_NO in {0..1}; do
INPUTFILE=${INPUTFILES[$TEST_RUN_NO]}
echo "Using file: ${INPUTFILE} , Running in: ${SCRAM_TEST_PATH} ."
(cmsDriver.py testExpressPPSAlCaRecoProducer -s ALCAPRODUCER:PPSCalMaxTracks,ENDJOB \
--process ALCARECO \
--scenario pp \
--era ctpps_2018 \
--conditions auto:run3_data_express \
--data \
--datatier ALCARECO \
--eventcontent ALCARECO \
--nThreads 8 \
--number 100 --filein ${INPUTFILE} \
--fileout file:outputALCAPPS_RECO_express.root \
--customise_commands="$customise_commands") || die 'failed running test_express_AlCaRecoProducer' $?
else
die "SKIPPING test, file ${INPUTFILE} not found" 0
fi
--process ALCARECO \
--scenario pp \
--era run3_common \
--conditions auto:run3_data_express \
--data \
--datatier ALCARECO \
--eventcontent ALCARECO \
--nThreads 8 \
--number 100 --filein ${INPUTFILE} \
--fileout file:outputALCAPPS_RECO_express_test${TEST_RUN_NO}.root \
--customise_commands="$COMMON_CUSTOM") || die 'failed running test_express_AlCaRecoProducer' $?
done
Original file line number Diff line number Diff line change
@@ -1,13 +1,25 @@
import FWCore.ParameterSet.Config as cms

# load era modifier to run on 2018 data
from Configuration.Eras.Modifier_ctpps_2018_cff import ctpps_2018
# load era modifier to run on 2022 data
from Configuration.Eras.Modifier_ctpps_2022_cff import ctpps_2022

process = cms.Process( 'TEST',ctpps_2018)
process = cms.Process( 'TEST',ctpps_2022)

# command line options
import FWCore.ParameterSet.VarParsing as VarParsing
options = VarParsing.VarParsing('analysis')
options.register('runNo',
1,
VarParsing.VarParsing.multiplicity.singleton,
VarParsing.VarParsing.varType.int,
"Run number")
options.parseArguments()

run_no = options.runNo

# LHCInfo plotter
process.load('Validation.CTPPS.ctppsLHCInfoPlotter_cfi')
process.ctppsLHCInfoPlotter.outputFile = "alcareco_lhc_info_express.root"
process.ctppsLHCInfoPlotter.outputFile = f"alcareco_lhc_info_express_{run_no}.root"

# Load geometry from DB
process.load("Geometry.VeryForwardGeometry.geometryRPFromDB_cfi")
Expand All @@ -21,7 +33,7 @@
rpId_56_N = cms.uint32(103),
rpId_56_F = cms.uint32(123),

outputFile = cms.string("alcareco_tracks_express.root")
outputFile = cms.string(f"alcareco_tracks_express_{run_no}.root")
)

# reconstruction plotter
Expand All @@ -35,7 +47,7 @@
rpId_56_N = cms.uint32(103),
rpId_56_F = cms.uint32(123),

outputFile = cms.string("alcareco_protons_express.root")
outputFile = cms.string(f"alcareco_protons_express_{run_no}.root")
)

process.p = cms.Path(
Expand All @@ -55,7 +67,7 @@

process.source = cms.Source( 'PoolSource',
fileNames = cms.untracked.vstring(
'file:outputALCAPPS_RECO_express.root',
options.inputFiles,
),
inputCommands = cms.untracked.vstring(
'keep *'
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/bin/bash
function die { echo $1: status $2; exit $2; }

(cmsRun ${SCRAM_TEST_PATH}/test_express_PPSAlCaReco_output.py) || die 'failed running test_express_PPSAlCaReco_output.py' $?
cmsRun ${SCRAM_TEST_PATH}/test_express_PPSAlCaReco_output.py inputFiles=file:outputALCAPPS_RECO_express_test0.root runNo=355207 || die 'failed running test_express_PPSAlCaReco_output.py' $?
cmsRun ${SCRAM_TEST_PATH}/test_express_PPSAlCaReco_output.py inputFiles=file:outputALCAPPS_RECO_express_test1.root runNo=367104 || die 'failed running test_express_PPSAlCaReco_output.py' $?
Original file line number Diff line number Diff line change
@@ -1,31 +1,32 @@
#!/bin/bash
function die { echo $1: status $2; exit $2; }

customise_commands="process.GlobalTag.toGet = cms.VPSet()\
\nprocess.GlobalTag.toGet.append(cms.PSet(record = cms.string(\"AlCaRecoTriggerBitsRcd\"),tag = cms.string(\"AlCaRecoHLTpaths_PPS2022_prompt_v1\"), connect = cms.string(\"frontier://FrontierProd/CMS_CONDITIONS\")))\
\nprocess.GlobalTag.toGet.append(cms.PSet(record = cms.string(\"PPSTimingCalibrationLUTRcd\"),tag = cms.string(\"PPSDiamondTimingCalibrationLUT_test\"), connect = cms.string(\"frontier://FrontierProd/CMS_CONDITIONS\")))\
\nprocess.ALCARECOPPSCalMaxTracksFilter.TriggerResultsTag = cms.InputTag(\"TriggerResults\",\"\",\"HLTX\")"
# customisation command needed for all tests cases
COMMON_CUSTOM="process.ALCARECOPPSCalMaxTracksFilter.TriggerResultsTag = cms.InputTag(\"TriggerResults\",\"\",\"HLTX\")"

INPUTFILE="/store/group/alca_global/pps_alcareco_producer_tests/outputALCAPPS_single.root"
COMMMAND=`xrdfs cms-xrd-global.cern.ch locate $INPUTFILE`
STATUS=$?
echo "xrdfs command status = "$STATUS
if [ $STATUS -eq 0 ]; then
echo "Using file ${INPUTFILE}. Running in ${SCRAM_TEST_PATH}."
# note we currently use `auto:run3_data_express` GT
# the correct GT (auto:run3_data_prompt) doesn't have LHCInfo record for run 322022 which corresponds to our face ALCARAW file
(cmsDriver.py testPromptPPSAlCaRecoProducer -s ALCAPRODUCER:PPSCalMaxTracks,ENDJOB \
--process ALCARECO \
--scenario pp \
--era ctpps_2018 \
--conditions auto:run3_data_express \
--data \
--datatier ALCARECO \
--eventcontent ALCARECO \
--nThreads 8 \
--number 100 --filein ${INPUTFILE} \
--fileout file:outputALCAPPS_RECO_prompt.root \
--customise_commands="$customise_commands") || die 'failed running test_prompt_AlCaRecoProducer' $?
else
die "SKIPPING test, file ${INPUTFILE} not found" 0
fi
# test on 2022 data
INPUTFILE_355207="/store/data/Run2022B/AlCaPPS/RAW/v1/000/355/207/00000/c23440f4-49c0-44aa-b8f6-f40598fb4705.root"

# new test on generated data, with same structure as data expected in 2024
INPUTFILE_367104="/store/group/alca_global/pps_alcareco_producer_tests/outputALCAPPSPrompt.root"

# all input files and customisation commands to loop through
INPUTFILES=($INPUTFILE_355207 $INPUTFILE_367104)

# test case loop
for TEST_RUN_NO in {0..1}; do
INPUTFILE=${INPUTFILES[$TEST_RUN_NO]}
echo "Using file: ${INPUTFILE} , Running in: ${SCRAM_TEST_PATH} ."
(cmsDriver.py testpromptPPSAlCaRecoProducer -s ALCAPRODUCER:PPSCalMaxTracks,ENDJOB \
--process ALCARECO \
--scenario pp \
--era run3_common \
--conditions auto:run3_data_prompt \
--data \
--datatier ALCARECO \
--eventcontent ALCARECO \
--nThreads 8 \
--number 100 --filein ${INPUTFILE} \
--fileout file:outputALCAPPS_RECO_prompt_test${TEST_RUN_NO}.root \
--customise_commands="$COMMON_CUSTOM") || die 'failed running test_prompt_AlCaRecoProducer' $?
done
Original file line number Diff line number Diff line change
@@ -1,13 +1,25 @@
import FWCore.ParameterSet.Config as cms

# load era modifier to run on 2018 data
from Configuration.Eras.Modifier_ctpps_2018_cff import ctpps_2018
# load era modifier to run on 2022 data
from Configuration.Eras.Modifier_ctpps_2022_cff import ctpps_2022

process = cms.Process( 'TEST',ctpps_2018)
process = cms.Process( 'TEST',ctpps_2022)

# command line options
import FWCore.ParameterSet.VarParsing as VarParsing
options = VarParsing.VarParsing('analysis')
options.register('runNo',
1,
VarParsing.VarParsing.multiplicity.singleton,
VarParsing.VarParsing.varType.int,
"Run number")
options.parseArguments()

run_no = options.runNo

# LHCInfo plotter
process.load('Validation.CTPPS.ctppsLHCInfoPlotter_cfi')
process.ctppsLHCInfoPlotter.outputFile = "alcareco_lhc_info_prompt.root"
process.ctppsLHCInfoPlotter.outputFile = f"alcareco_lhc_info_prompt_{run_no}.root"

# Load geometry from DB
process.load("Geometry.VeryForwardGeometry.geometryRPFromDB_cfi")
Expand All @@ -21,7 +33,7 @@
rpId_56_N = cms.uint32(103),
rpId_56_F = cms.uint32(123),

outputFile = cms.string("alcareco_tracks_prompt.root")
outputFile = cms.string(f"alcareco_tracks_prompt_{run_no}.root")
)

# reconstruction plotter
Expand All @@ -35,7 +47,7 @@
rpId_56_N = cms.uint32(103),
rpId_56_F = cms.uint32(123),

outputFile = cms.string("alcareco_protons_prompt.root")
outputFile = cms.string(f"alcareco_protons_prompt_{run_no}.root")
)

process.p = cms.Path(
Expand All @@ -51,12 +63,11 @@
# load GT
process.load('Configuration.StandardSequences.FrontierConditions_GlobalTag_cff')
from Configuration.AlCa.GlobalTag import GlobalTag
process.GlobalTag = GlobalTag(process.GlobalTag, 'auto:run3_data_express')
# process.GlobalTag = GlobalTag(process.GlobalTag, 'auto:run3_data_prompt') # --> No LHCInfo, temporarily using the express GT
process.GlobalTag = GlobalTag(process.GlobalTag, 'auto:run3_data_prompt')

process.source = cms.Source( 'PoolSource',
fileNames = cms.untracked.vstring(
'file:outputALCAPPS_RECO_prompt.root',
options.inputFiles,
),
inputCommands = cms.untracked.vstring(
'keep *'
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/bin/bash
function die { echo $1: status $2; exit $2; }

(cmsRun ${SCRAM_TEST_PATH}/test_prompt_PPSAlCaReco_output.py) || die 'failed running test_prompt_PPSAlCaReco_output.py' $?
cmsRun ${SCRAM_TEST_PATH}/test_prompt_PPSAlCaReco_output.py inputFiles=file:outputALCAPPS_RECO_prompt_test0.root runNo=355207 || die 'failed running test_prompt_PPSAlCaReco_output.py' $?
cmsRun ${SCRAM_TEST_PATH}/test_prompt_PPSAlCaReco_output.py inputFiles=file:outputALCAPPS_RECO_prompt_test1.root runNo=367104 || die 'failed running test_prompt_PPSAlCaReco_output.py' $?
1 change: 1 addition & 0 deletions Configuration/PyReleaseValidation/python/relval_steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -2424,6 +2424,7 @@ def lhegensim2018ml(fragment,howMuch):
'--data': '',
'--datatier':'ALCARECO',
'--eventcontent':'ALCARECO',
'--customise_commands':'"process.ctppsRawToDigiTaskAlCaRecoProducer = cms.Task(process.ctppsDiamondRawToDigiAlCaRecoProducer, process.totemTimingRawToDigiAlCaRecoProducer, process.ctppsPixelDigisAlCaRecoProducer)"' # disable gtStage2DigisAlCaRecoProducer as 2022 data used in this workflow doesn't have necessary products
}

steps['TIER0EXPPPSCALALIG']={'-s':'RAW2DIGI,L1Reco,ALCAPRODUCER:PPSCalTrackBasedSel,ENDJOB',
Expand Down

0 comments on commit 6e62922

Please sign in to comment.