Move "hybrid" tau production from nano to mini step #46196

mbluj · 2024-10-02T10:14:53Z

PR description:

This PR moves production of hybrid-taus, i.e. collection of tau candidates reconstructed by the HPS algorithm and jet tagged by unified jet taggers as tau-like but not reconstructed by HPS, from nanoAOD to miniAOD step. This movement unfies tau information available in both mini and nano data tiers.

PR validation:

Tested with custom re-mini+nano and with nano workflows.
Matrix tests (limited) successful.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport is not planned.

cmsbuild · 2024-10-02T10:15:19Z

cms-bot internal usage

cmsbuild · 2024-10-02T10:16:33Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46196/41996

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/Eras/python/Modifier_run3_nanoAOD_pre142X_cff.py modified in PR(s): Boosted DeepTauID v2.0 #46112
- File Configuration/StandardSequences/python/Eras.py modified in PR(s): Add run3 tracking low pu era #33532, Era for UPC re-reco of 2016 pPb data #46087, Boosted DeepTauID v2.0 #46112, Remove run3_nanoAOD_122 & 124 modifiers #46151
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): Adding PAIReDJet Table to NanoAOD #45207, Boosted DeepTauID v2.0 #46112, Remove run3_nanoAOD_122 & 124 modifiers #46151
- File PhysicsTools/NanoAOD/python/nano_eras_cff.py modified in PR(s): Boosted DeepTauID v2.0 #46112, Remove run3_nanoAOD_122 & 124 modifiers #46151
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): Boosted DeepTauID v2.0 #46112

cmsbuild · 2024-10-02T10:16:56Z

A new Pull Request was created by @mbluj for master.

It involves the following packages:

Configuration/Eras (operations)
Configuration/StandardSequences (operations)
PhysicsTools/NanoAOD (xpog)
PhysicsTools/PatAlgos (reconstruction, xpog)

@antoniovilela, @cmsbuild, @davidlange6, @fabiocos, @ftorrresd, @hqucms, @jfernan2, @mandrenguyen, @rappoccio can you please review it and eventually sign? Thanks.
@AlexDeMoor, @AnnikaStein, @GiacomoSguazzoni, @JanFSchulte, @Martin-Grunewald, @Ming-Yan, @Senphy, @VinInn, @VourMa, @ahinzmann, @andrzejnovak, @azotz, @castaned, @demuller, @dgulhan, @fabiocos, @felicepantaleo, @gkasieczka, @gouskos, @gpetruc, @hatakeyamak, @jdamgov, @jdolen, @makortel, @mariadalfonso, @mbluj, @missirol, @mmarionncern, @mmusich, @mtosi, @nhanvtran, @rappoccio, @rovere, @sameasy, @schoef, @seemasharmafnal, @slomeo this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

hqucms · 2024-10-02T10:23:20Z

Hi @mbluj -- Can you rebase as #46066 has already been merged? Thanks!

hqucms · 2024-10-02T10:26:10Z

enable nano

mbluj · 2024-10-02T10:59:01Z

Hi @mbluj -- Can you rebase as #46066 has already been merged? Thanks!

In meantime I have found an unexpected issue when running matrix tests - I will fix it and then rebase.

cmsbuild · 2024-10-02T14:30:11Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46196/42005

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/Eras/python/Modifier_run3_nanoAOD_pre142X_cff.py modified in PR(s): Boosted DeepTauID v2.0 #46112
- File Configuration/StandardSequences/python/Eras.py modified in PR(s): Add run3 tracking low pu era #33532, Era for UPC re-reco of 2016 pPb data #46087, Boosted DeepTauID v2.0 #46112, Remove run3_nanoAOD_122 & 124 modifiers #46151
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): Adding PAIReDJet Table to NanoAOD #45207, Boosted DeepTauID v2.0 #46112, Remove run3_nanoAOD_122 & 124 modifiers #46151
- File PhysicsTools/NanoAOD/python/nano_eras_cff.py modified in PR(s): Boosted DeepTauID v2.0 #46112, Remove run3_nanoAOD_122 & 124 modifiers #46151
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): Boosted DeepTauID v2.0 #46112

cmsbuild · 2024-10-02T14:30:34Z

Pull request #46196 was updated. @antoniovagnerini, @antoniovilela, @cmsbuild, @davidlange6, @fabiocos, @ftorrresd, @hqucms, @jfernan2, @mandrenguyen, @nothingface0, @rappoccio, @rvenditti, @syuvivida, @tjavaid can you please check and sign again.

cmsbuild · 2024-10-02T14:42:33Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46196/42007

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/Eras/python/Modifier_run3_nanoAOD_pre142X_cff.py modified in PR(s): Boosted DeepTauID v2.0 #46112
- File Configuration/StandardSequences/python/Eras.py modified in PR(s): Add run3 tracking low pu era #33532, Era for UPC re-reco of 2016 pPb data #46087, Boosted DeepTauID v2.0 #46112, Remove run3_nanoAOD_122 & 124 modifiers #46151
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): Adding PAIReDJet Table to NanoAOD #45207, Boosted DeepTauID v2.0 #46112, Remove run3_nanoAOD_122 & 124 modifiers #46151
- File PhysicsTools/NanoAOD/python/nano_eras_cff.py modified in PR(s): Boosted DeepTauID v2.0 #46112, Remove run3_nanoAOD_122 & 124 modifiers #46151
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): Boosted DeepTauID v2.0 #46112
- File Validation/RecoTau/src/TauValidationMiniAOD.cc modified in PR(s): Tau DQM miniAOD validation for DATA #44029

mbluj · 2024-10-08T14:49:04Z

FYI, nothing really changed, it was just a routine rebasing to the current IB.

cmsbuild · 2024-10-08T16:27:23Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46196/42127

There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): Adding PAIReDJet Table to NanoAOD #45207, Boosted DeepTauID v2.0 #46112
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): Boosted DeepTauID v2.0 #46112
- File Validation/RecoTau/src/TauValidationMiniAOD.cc modified in PR(s): Tau DQM miniAOD validation for DATA #44029

cmsbuild · 2024-10-08T16:27:45Z

Pull request #46196 was updated. @antoniovagnerini, @civanch, @cmsbuild, @ftorrresd, @hqucms, @jfernan2, @kpedro88, @mandrenguyen, @mdhildreth, @nothingface0, @rvenditti, @syuvivida, @tjavaid can you please check and sign again.

mbluj · 2024-10-08T16:39:36Z

Last commits should fix issues with unit tests

cmsbuild · 2024-10-08T16:41:27Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46196/42128

There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): Adding PAIReDJet Table to NanoAOD #45207, Boosted DeepTauID v2.0 #46112
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): Boosted DeepTauID v2.0 #46112
- File Validation/RecoTau/src/TauValidationMiniAOD.cc modified in PR(s): Tau DQM miniAOD validation for DATA #44029

cmsbuild · 2024-10-08T16:41:47Z

Pull request #46196 was updated. @antoniovagnerini, @civanch, @cmsbuild, @ftorrresd, @hqucms, @jfernan2, @kpedro88, @mandrenguyen, @mdhildreth, @nothingface0, @rvenditti, @syuvivida, @tjavaid can you please check and sign again.

hqucms · 2024-10-08T17:51:29Z

please test

cmsbuild · 2024-10-08T21:21:18Z

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3acba5/42051/summary.html
COMMIT: 18885f8
CMSSW: CMSSW_14_2_X_2024-10-08-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46196/42051/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3acba5/42051/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3acba5/42051/git-merge-result

Comparison Summary

Summary:

You potentially added 8 lines to the logs
Reco comparison results: 1218 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3331066
DQMHistoTests: Total failures: 3820
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3327226
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 193 log files, 163 edm output root files, 44 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially removed 116 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 335 differences found in the comparisons
DQMHistoTests: Total files compared: 21
DQMHistoTests: Total histograms compared: 55028
DQMHistoTests: Total failures: 244
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 54784
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 20 files compared)
Checked 102 log files, 58 edm output root files, 21 DQM output files
TriggerResults: no differences found

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.001	2.947	2.947	0.000 ( +0.0% )	3.06	3.11	-1.7%	6.688	6.691
2500.002	3.056	3.056	0.000 ( +0.0% )	2.74	2.76	-0.6%	7.045	7.041
2500.003	2.998	2.998	0.000 ( +0.0% )	2.84	2.90	-2.0%	7.013	7.015
2500.011	1.533	1.533	0.000 ( +0.0% )	4.68	4.81	-2.7%	2.424	2.421
2500.012	2.030	2.030	0.000 ( +0.0% )	2.87	2.91	-1.2%	2.611	2.610
2500.013	1.872	1.872	0.000 ( +0.0% )	4.04	4.14	-2.5%	2.520	2.515
2500.021	0.022	0.022	0.000 ( +0.0% )	0.93	0.99	-5.4%	2.386	2.384
2500.022	0.022	0.022	0.000 ( +0.0% )	0.90	0.94	-4.7%	2.380	2.379
2500.023	0.022	0.022	0.000 ( +0.0% )	0.89	0.95	-6.2%	2.248	2.247
2500.024	0.022	0.022	0.000 ( +0.2% )	0.67	0.71	-6.5%	2.469	2.467
2500.031	0.035	0.035	0.000 ( +0.0% )	0.82	0.85	-3.5%	2.444	2.443
2500.032	0.036	0.036	0.000 ( +0.0% )	0.83	0.88	-5.7%	2.410	2.409
2500.033	0.037	0.037	0.000 ( +0.0% )	0.75	0.80	-6.2%	2.491	2.487
2500.034	0.036	0.036	0.000 ( +0.0% )	0.74	0.82	-9.1%	2.470	2.469
2500.101	2.728	2.728	0.000 ( +0.0% )	8.48	8.97	-5.4%	6.960	6.960
2500.111	1.386	1.386	0.000 ( +0.0% )	18.43	19.49	-5.4%	2.233	2.227
2500.112	1.803	1.803	0.000 ( +0.0% )	14.28	14.97	-4.6%	2.309	2.309
2500.131	0.747	0.747	0.000 ( +0.0% )	17.26	18.45	-6.4%	1.502	1.498
2500.201	2.552	2.552	0.000 ( +0.0% )	7.32	7.74	-5.3%	6.239	6.235
2500.211	1.699	1.699	0.000 ( +0.0% )	17.29	17.99	-3.9%	2.281	2.279
2500.212	2.095	2.095	0.000 ( +0.0% )	13.46	14.48	-7.0%	2.367	2.364
2500.221	1.998	1.998	0.000 ( +0.0% )	7.48	7.90	-5.3%	2.455	2.452
2500.222	3.276	3.276	0.000 ( +0.0% )	7.08	7.76	-8.7%	2.536	2.542
2500.223	8.963	8.963	0.000 ( +0.0% )	2.49	2.70	-7.7%	2.565	2.563
2500.224	5.810	5.810	0.000 ( +0.0% )	0.59	0.60	-2.4%	2.605	2.599
2500.225	5.827	5.827	0.000 ( +0.0% )	0.57	0.59	-3.1%	2.625	2.626
2500.226	3.038	3.038	0.000 ( +0.0% )	7.04	7.71	-8.7%	2.540	2.535
2500.227	1.437	1.437	0.000 ( +0.0% )	11.26	12.00	-6.1%	1.437	1.433
2500.231	1.403	1.403	0.000 ( +0.0% )	13.43	14.59	-8.0%	2.190	2.182
2500.232	2.300	2.300	0.000 ( +0.0% )	13.31	14.27	-6.8%	2.277	2.271
2500.233	4.747	4.747	0.000 ( +0.0% )	4.02	4.40	-8.6%	2.293	2.287
2500.234	3.518	3.518	0.000 ( +0.0% )	0.75	0.77	-2.4%	2.311	2.315
2500.235	3.530	3.530	0.000 ( +0.0% )	0.74	0.75	-1.4%	2.339	2.329
2500.236	2.145	2.145	0.000 ( +0.0% )	13.65	14.36	-5.0%	2.273	2.266
2500.237	1.016	1.016	0.000 ( +0.0% )	14.67	17.38	-15.6%	1.476	1.471
2500.241	9.404	9.404	0.000 ( +0.0% )	3.62	3.72	-2.5%	1.934	1.946
2500.242	10.331	10.331	0.000 ( +0.0% )	0.85	0.90	-5.9%	1.740	1.734
2500.243	2.712	2.712	0.000 ( +0.0% )	7.96	8.74	-8.9%	1.077	1.075
2500.244	485.976	485.976	0.000 ( +0.0% )	0.53	0.57	-7.7%	1.671	1.681
2500.245	823.202	823.202	0.000 ( +0.0% )	0.70	0.74	-5.2%	1.663	1.655
2500.901	1.777	1.777	0.000 ( +0.0% )	19.37	21.38	-9.4%	1.841	1.839
2500.902	1.626	1.626	0.000 ( +0.0% )	20.73	21.34	-2.9%	1.776	1.768
2500.911	13.995	13.995	0.000 ( +0.0% )	2.34	3.27	-28.5%	1.100	1.095
2500.912	0.438	0.171	0.268 ( +157.1% )	1.25	1.60	-22.1%	0.975	0.974
2500.913	0.110	0.110	0.000 ( +0.0% )	1.00	1.08	-7.7%	0.977	0.975

hqucms · 2024-10-14T16:47:44Z

+1

jfernan2 · 2024-10-14T17:56:17Z

+1

cmsbuild added this to the CMSSW_14_2_X milestone Oct 2, 2024

cmsbuild added reconstruction-pending operations-pending pending-signatures tests-pending orp-pending code-checks-pending xpog-pending labels Oct 2, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 2, 2024

cmsbuild added dqm-pending code-checks-pending and removed code-checks-approved labels Oct 2, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 2, 2024

mbluj force-pushed the CMSSW_14_2_X_hybridTauAtMini branch from 2f97a10 to a62be2b Compare October 2, 2024 14:41

cmsbuild added code-checks-pending and removed operations-pending code-checks-approved labels Oct 2, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 2, 2024

synchronise tau-embedding tests with current producton setup

fc5d678

cmsbuild added simulation-pending code-checks-pending and removed code-checks-approved labels Oct 8, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 8, 2024

Merge branch 'master' into CMSSW_14_2_X_hybridTauAtMini

18885f8

cmsbuild added code-checks-pending and removed code-checks-approved labels Oct 8, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 8, 2024

cmsbuild mentioned this pull request Oct 8, 2024

Boosted DeepTauID v2.0 #46112

Open

cmsbuild added tests-started and removed tests-pending labels Oct 8, 2024

cmsbuild added tests-approved and removed tests-started labels Oct 8, 2024

cmsbuild added xpog-approved and removed xpog-pending labels Oct 14, 2024

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Oct 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move "hybrid" tau production from nano to mini step #46196

Move "hybrid" tau production from nano to mini step #46196

mbluj commented Oct 2, 2024 •

edited

Loading

cmsbuild commented Oct 2, 2024 •

edited

Loading

cmsbuild commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

hqucms commented Oct 2, 2024

hqucms commented Oct 2, 2024

mbluj commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

mbluj commented Oct 8, 2024

cmsbuild commented Oct 8, 2024

cmsbuild commented Oct 8, 2024

mbluj commented Oct 8, 2024

cmsbuild commented Oct 8, 2024

cmsbuild commented Oct 8, 2024

hqucms commented Oct 8, 2024

cmsbuild commented Oct 8, 2024

hqucms commented Oct 14, 2024

jfernan2 commented Oct 14, 2024

Move "hybrid" tau production from nano to mini step #46196

Are you sure you want to change the base?

Move "hybrid" tau production from nano to mini step #46196

Conversation

mbluj commented Oct 2, 2024 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Oct 2, 2024 • edited Loading

cmsbuild commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

hqucms commented Oct 2, 2024

hqucms commented Oct 2, 2024

mbluj commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

mbluj commented Oct 8, 2024

cmsbuild commented Oct 8, 2024

cmsbuild commented Oct 8, 2024

mbluj commented Oct 8, 2024

cmsbuild commented Oct 8, 2024

cmsbuild commented Oct 8, 2024

hqucms commented Oct 8, 2024

cmsbuild commented Oct 8, 2024

Comparison Summary

NANO Comparison Summary

hqucms commented Oct 14, 2024

jfernan2 commented Oct 14, 2024

mbluj commented Oct 2, 2024 •

edited

Loading

cmsbuild commented Oct 2, 2024 •

edited

Loading