Update Alpaka to version 1.1.0 #43772

fwyzard · 2024-01-23T13:25:05Z

PR description:

Update the CMSSW utilities for Alpaka v1.1.0.

This is required to go together with cms-sw/cmsdist#8957 .

PR validation:

Unit tests pass.

cmsbuild · 2024-01-23T13:25:31Z

cms-bot internal usage

cmsbuild · 2024-01-23T13:32:05Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43772/38502

This PR adds an extra 28KB to repository

cmsbuild · 2024-01-23T13:32:27Z

A new Pull Request was created by @fwyzard (Andrea Bocci) for master.

It involves the following packages:

HeterogeneousCore/AlpakaCore (heterogeneous)
HeterogeneousCore/AlpakaInterface (heterogeneous)

@cmsbuild, @makortel, @fwyzard can you please review it and eventually sign? Thanks.
@missirol, @makortel, @rovere this is something you requested to watch as well.
@sextonkennedy, @antoniovilela, @rappoccio you are the release manager for this.

cms-bot commands are listed here

fwyzard · 2024-01-23T14:40:31Z

enable gpu

fwyzard · 2024-01-23T14:40:33Z

please test with cms-sw/cmsdist/pull/8957

fwyzard · 2024-01-23T14:40:43Z

+heterogeneous

cmsbuild · 2024-01-23T14:46:11Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43772/38508

This PR adds an extra 48KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File HeterogeneousCore/AlpakaInterface/interface/HistoContainer.h modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaInterface/interface/OneToManyAssoc.h modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaInterface/interface/prefixScan.h modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaInterface/interface/radixSort.h modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaInterface/test/alpaka/testHistoContainer.dev.cc modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaInterface/test/alpaka/testOneHistoContainer.dev.cc modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaInterface/test/alpaka/testOneToManyAssoc.dev.cc modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaInterface/test/alpaka/testPrefixScan.dev.cc modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117

cmsbuild · 2024-01-23T14:46:35Z

Pull request #43772 was updated. can you please check and sign again.

cmsbuild · 2024-01-23T15:01:18Z

-1

Failed Tests: Build HeaderConsistency ClangBuild
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-842e74/36991/summary.html
COMMIT: da13c59
CMSSW: CMSSW_14_0_X_2024-01-23-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43772/36991/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@ericcano Ports prefixScan, OneToManyAssoc and HistoContainer from CUDAUtilities. #43064

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-842e74/36991/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-842e74/36991/git-merge-result

Build

I found compilation error when building:

cp: cannot stat 'tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestKernelROCmAsync/libalpakaTestKernelROCmAsync_rocm.a': No such file or directory
cp: cannot stat 'tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestIndependentKernelROCmAsync/libalpakaTestIndependentKernelROCmAsync_rocm.a': No such file or directory
Copying tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestRadixSortROCmAsync/libalpakaTestRadixSortROCmAsync_rocm.a to productstore area:
cp: cannot stat 'tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestOneHistoContainerROCmAsync/libalpakaTestOneHistoContainerROCmAsync_rocm.a': No such file or directory
>> Deleted: tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestHistoContainerROCmAsync/libalpakaTestHistoContainerROCmAsync_rocm.a
gmake: *** [config/SCRAM/GMake/Makefile.rules:1820: tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestHistoContainerROCmAsync/libalpakaTestHistoContainerROCmAsync_rocm.a] Error 1
Copying tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestSimpleVectorROCmAsync/libalpakaTestSimpleVectorROCmAsync_rocm.a to productstore area:
cp: cannot stat 'tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestOneRadixSortROCmAsync/libalpakaTestOneRadixSortROCmAsync_rocm.a': No such file or directory
Copying tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaWorkDivisionROCmAsync/libalpakaWorkDivisionROCmAsync_rocm.a to productstore area:
>> Deleted: tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestAtomicPairCounterROCmAsync/libalpakaTestAtomicPairCounterROCmAsync_rocm.a
gmake: *** [config/SCRAM/GMake/Makefile.rules:1820: tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestAtomicPairCounterROCmAsync/libalpakaTestAtomicPairCounterROCmAsync_rocm.a] Error 1

Clang Build

I found compilation error while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 32 COMPILER='llvm compile'

>> Creating project symlinks
>> Entering Package HeterogeneousCore/AlpakaCore
>> Entering Package HeterogeneousCore/AlpakaInterface
>> Entering Package HeterogeneousCore/CUDAUtilities
>> Compile sequence completed for CMSSW CMSSW_14_0_X_2024-01-23-1100
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 1
+ eval scram build outputlog '&&' '(python3' /data/cmsbld/jenkins/workspace/ib-run-pr-tests/cms-bot/buildLogAnalyzer.py --logDir /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_14_0_X_2024-01-23-1100/tmp/el8_amd64_gcc12/cache/log/src '||' 'true)'
++ scram build outputlog
>> Entering Package HeterogeneousCore/AlpakaCore
Entering library rule at src/HeterogeneousCore/AlpakaCore/plugins
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_14_0_X_2024-01-23-1100/src/HeterogeneousCore/AlpakaCore/plugins/ModuleTypeResolverAlpaka.cc

fwyzard · 2024-02-12T20:32:28Z

+heterogeneous

fwyzard · 2024-02-12T20:34:04Z

please test with cms-sw/cmsdist#8957

fwyzard · 2024-02-12T20:39:05Z

@cms-sw/reconstruction-l2 The RECO changes should be trivial (ALPAKA_ASSERT_OFFLOAD renamed to ALPAKA_ASSERT_ACC).

cmsbuild · 2024-02-12T21:56:53Z

REMINDER @antoniovilela, @sextonkennedy, @rappoccio: This PR was tested with cms-sw/cmsdist#8957, please check if they should be merged together

fwyzard · 2024-02-12T23:00:24Z

REMINDER @antoniovilela, @sextonkennedy, @rappoccio: This PR was tested with cms-sw/cmsdist#8957, please check if they should be merged together

Yes, they should be merged together :-)

cmsbuild · 2024-02-13T06:39:42Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-842e74/37385/summary.html
COMMIT: 856ec61
CMSSW: CMSSW_14_1_X_2024-02-12-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43772/37385/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-842e74/37385/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-842e74/37385/git-merge-result

Comparison Summary

Summary:

You potentially removed 93 lines from the logs
Reco comparison results: 58 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3247438
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3247410
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 19061.808 KiB( 47 files compared)
DQMHistoSizes: changed ( 23234.0,... ): 14102.535 KiB TrackerPhase2OTStub/Stubs
DQMHistoSizes: changed ( 23234.0,... ): -14102.286 KiB SiOuterTracker/Stubs
DQMHistoSizes: changed ( 23234.0,... ): 14027.626 KiB TrackerPhase2TTCluster/Clusters
DQMHistoSizes: changed ( 23234.0,... ): -14027.360 KiB SiOuterTracker/Clusters
DQMHistoSizes: changed ( 23234.0,... ): 3178.168 KiB TrackerPhase2OTStubV/Stub_RZ
DQMHistoSizes: changed ( 23234.0,... ): -25.679 KiB SiOuterTrackerV/Tracks
DQMHistoSizes: changed ( 23234.0,... ): 20.667 KiB TrackerPhase2OTL1TrackV/Resolution
DQMHistoSizes: changed ( 23234.0,... ): 16.476 KiB TrackerPhase2OTL1Track/Tracks
DQMHistoSizes: changed ( 23234.0,... ): -16.163 KiB SiOuterTracker/Tracks
DQMHistoSizes: changed ( 23234.0,... ): 2.961 KiB TrackerPhase2OTL1TrackV/Efficiency
DQMHistoSizes: changed ( 23234.0 ): ...
Checked 200 log files, 161 edm output root files, 48 DQM output files
TriggerResults: found differences in 5 / 46 workflows

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 32 differences found in the comparisons
DQMHistoTests: Total files compared: 3
DQMHistoTests: Total histograms compared: 39740
DQMHistoTests: Total failures: 1385
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 38355
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 2 files compared)
Checked 8 log files, 10 edm output root files, 3 DQM output files
TriggerResults: no differences found

jfernan2 · 2024-02-13T11:34:43Z

+1

cmsbuild · 2024-02-13T11:35:10Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @sextonkennedy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)
Notice This PR was tested with additional Pull Request(s), please also merge them if necessary: cms-sw/cmsdist#8957

antoniovilela · 2024-02-13T13:06:27Z

+1

cmsbuild added this to the CMSSW_14_0_X milestone Jan 23, 2024

cmsbuild added pending-signatures tests-pending orp-pending code-checks-pending heterogeneous-pending labels Jan 23, 2024

fwyzard mentioned this pull request Jan 23, 2024

Update Alpaka to version 1.1.0 cms-sw/cmsdist#8957

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 23, 2024

fwyzard force-pushed the cmssw_alpaka_v1.1.0 branch from 155cb21 to da13c59 Compare January 23, 2024 14:39

cmsbuild added code-checks-pending and removed code-checks-approved labels Jan 23, 2024

cmsbuild added fully-signed tests-started heterogeneous-approved and removed pending-signatures tests-pending heterogeneous-pending labels Jan 23, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 23, 2024

cmsbuild removed the tests-started label Jan 23, 2024

cmsbuild removed db-pending geometry-pending pdmv-pending upgrade-pending labels Feb 12, 2024

cmsbuild added tests-started heterogeneous-approved and removed tests-pending heterogeneous-pending labels Feb 12, 2024

cmsbuild added the requires-external label Feb 12, 2024

fwyzard mentioned this pull request Feb 12, 2024

Update Alpaka to version 1.1.0 [14.0.x] #43943

Merged

cmsbuild added tests-approved and removed tests-started labels Feb 13, 2024

cmsbuild added reconstruction-approved fully-signed and removed reconstruction-pending pending-signatures labels Feb 13, 2024

cmsbuild added orp-approved and removed orp-pending labels Feb 13, 2024

cmsbuild merged commit a26bc2e into cms-sw:master Feb 13, 2024
24 checks passed

cmsbuild mentioned this pull request Feb 14, 2024

Phase-2 TauMinator Tau (NNCaloTau) #42840

Merged

fwyzard deleted the cmssw_alpaka_v1.1.0 branch March 2, 2024 22:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Alpaka to version 1.1.0 #43772

Update Alpaka to version 1.1.0 #43772

fwyzard commented Jan 23, 2024

cmsbuild commented Jan 23, 2024 •

edited

Loading

cmsbuild commented Jan 23, 2024

cmsbuild commented Jan 23, 2024

fwyzard commented Jan 23, 2024

fwyzard commented Jan 23, 2024

fwyzard commented Jan 23, 2024

cmsbuild commented Jan 23, 2024

cmsbuild commented Jan 23, 2024

cmsbuild commented Jan 23, 2024

fwyzard commented Feb 12, 2024

fwyzard commented Feb 12, 2024

fwyzard commented Feb 12, 2024

cmsbuild commented Feb 12, 2024

fwyzard commented Feb 12, 2024

cmsbuild commented Feb 13, 2024

jfernan2 commented Feb 13, 2024

cmsbuild commented Feb 13, 2024

antoniovilela commented Feb 13, 2024

Update Alpaka to version 1.1.0 #43772

Update Alpaka to version 1.1.0 #43772

Conversation

fwyzard commented Jan 23, 2024

PR description:

PR validation:

cmsbuild commented Jan 23, 2024 • edited Loading

cmsbuild commented Jan 23, 2024

cmsbuild commented Jan 23, 2024

fwyzard commented Jan 23, 2024

fwyzard commented Jan 23, 2024

fwyzard commented Jan 23, 2024

cmsbuild commented Jan 23, 2024

cmsbuild commented Jan 23, 2024

cmsbuild commented Jan 23, 2024

Build

Clang Build

fwyzard commented Feb 12, 2024

fwyzard commented Feb 12, 2024

fwyzard commented Feb 12, 2024

cmsbuild commented Feb 12, 2024

fwyzard commented Feb 12, 2024

cmsbuild commented Feb 13, 2024

Comparison Summary

GPU Comparison Summary

jfernan2 commented Feb 13, 2024

cmsbuild commented Feb 13, 2024

antoniovilela commented Feb 13, 2024

cmsbuild commented Jan 23, 2024 •

edited

Loading