Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to multi layout portable collections (up to 5) [14.0.x] #43946

Merged
merged 1 commit into from
Feb 14, 2024

Conversation

fwyzard
Copy link
Contributor

@fwyzard fwyzard commented Feb 12, 2024

PR description:

The design is recursive, but ROOT seems not to serialize variadic templates, so we have a 5 elements design. This number is arbitrary and can be changed with relatively little work.

The order of the PortableDeviceCollection template parameters is changed from device at the end to device at the beginning to accommodate the pseudo variadic templation.

A test validates the collection with two layouts and three layouts.

The layout and view can be accessed by index or type, on the condition the type is present only once in the collection.

The default index is set to 0 to minimize the need for changes in the single layout case. Empty template parameters square bracket ("<>") are needed in some cases.

Convenience scripts are added to generate the XML need for dictionary generation but has to be edited if there are commonalities between multiple collections.

PR validation:

The PR is validated with the pre-existing HeterogeneousCore/AlpakaTest/test/testHeterogeneousCoreAlpakaTestWriteRead.sh which has been extended to add a two-layouts example.

Backport status

Backport of #40285 to CMSSW 14.0.x, from the same branch.

The design is recursive, but ROOT seems not to serialize variadic templates, so we have a
5 elements design. This number is arbitrary and can be changed with relatively little work.

The order of the PortableDeviceCollection template parameters is changed from device at the end
to device at the begining to accomodate the pseudo variadic templation.

A test validates the colleciton with two layouts and three layouts.

The layout and view can be accessed by index or type, on the condition the type is present only
once in the collection.

The default index is set to 0 to minimize the need for chages in the single layout case. Empty
template parameters square bracket ("<>") are needed in some cases.

Convenience scripts are added to generate the XML need for dictionary generation but has to be edited
if there are commonalities between multiple collections.
@fwyzard
Copy link
Contributor Author

fwyzard commented Feb 12, 2024

please test

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @fwyzard (Andrea Bocci) for CMSSW_14_0_X.

It involves the following packages:

  • DataFormats/Portable (heterogeneous)
  • DataFormats/PortableTestObjects (heterogeneous)
  • HeterogeneousCore/AlpakaTest (heterogeneous)

@makortel, @fwyzard can you please review it and eventually sign? Thanks.
@rovere, @makortel, @missirol this is something you requested to watch as well.
@antoniovilela, @sextonkennedy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 12, 2024

cms-bot internal usage

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3a7af4/37394/summary.html
COMMIT: b4ed3a8
CMSSW: CMSSW_14_0_X_2024-02-12-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43946/37394/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test test-das-selected-lumis had ERRORS

Comparison Summary

Summary:

@fwyzard
Copy link
Contributor Author

fwyzard commented Feb 13, 2024

Unit Tests

I found 1 errors in the following unit tests:

---> test test-das-selected-lumis had ERRORS

The failed test is unrelated to this PR:

===== Test "test-das-selected-lumis" ====
+ export CMS_BOT_USE_DASGOCLIENT=true
+ CMS_BOT_USE_DASGOCLIENT=true
+ QUERY='lumi,file dataset=/HIHardProbes/HIRun2018A-v1/RAW run=326479'
+ dasgoclient --limit 0 --query 'lumi,file dataset=/HIHardProbes/HIRun2018A-v1/RAW run=326479' --format json
+ das-selected-lumis.py 1,23
+ grep '^/store/'
2024/02/13 03:25:10 ERROR failed to parse X509 proxy: crypto/tls: failed to parse key: asn1: syntax error: data truncated

---> test test-das-selected-lumis had ERRORS
TestTime:3
^^^^ End Test test-das-selected-lumis ^^^^

@smuzaffar
Copy link
Contributor

ignore tests-rejected with external-failure

we have randomly seen such failure but really do not understand why dasgoclient soemtimes can not parse the key

@fwyzard
Copy link
Contributor Author

fwyzard commented Feb 13, 2024

+heterogeneous

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_14_0_X IBs (test failures were overridden) and once validation in the development release cycle CMSSW_14_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @rappoccio, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

@fwyzard
Copy link
Contributor Author

fwyzard commented Feb 13, 2024

urgent

for CMSSW_14_0_0, as discussed at the ORP on 2024/02/13

@antoniovilela
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit e471502 into cms-sw:CMSSW_14_0_X Feb 14, 2024
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants