scram tool clean; support for building stack without an option tool #9218

smuzaffar · 2024-06-03T07:15:37Z

The PR proposes following two updates

Allow to have rpm macros in scram toolfiles. This allows to properly generate the toolfiles using the package informaton atbuild time instead of setting env or doing sed/grep in %post section. This is done by adding all tools in a single file and then expanding that file with in the actual toolfile or tool-conf
Support added to build the stack without optional tools e.g. on Risc-V we do not have rocm/cuda. So using new feature of pkgtools , now we should be able to build the external software tool stack for Risc-V.
cpu_features update to version 0.9.0 which has risc-v support/fixes

smuzaffar · 2024-06-03T07:16:05Z

test parameters:

pull_request = Support for building software stack with options packages e.g. cuda/rocm pkgtools#209
full_cmssw = true
enable = gpu

cmsbuild · 2024-06-03T07:16:06Z

A new Pull Request was created by @smuzaffar for branch IB/CMSSW_14_1_X/master.

@aandvalenzuela, @smuzaffar, @cmsbuild, @iarspider can you please review it and eventually sign? Thanks.
@sextonkennedy, @rappoccio, @antoniovilela you are the release manager for this.
cms-bot commands are listed here

cmsbuild · 2024-06-03T07:16:06Z

cms-bot internal usage

smuzaffar · 2024-06-03T07:16:10Z

please test

iarspider · 2024-06-03T07:33:35Z

onnxruntime.spec

@@ -21,7 +19,7 @@ rm -rf ../build; mkdir ../build; cd ../build
 USE_CUDA=OFF
 %if "%{cmsos}" != "slc7_aarch64"
 if [ "%{cuda_gcc_support}" = "true" ] ; then
-USE_CUDA=ON
+USE_CUDA=%{!?without_cuda:ON}


@smuzaffar Sholdn't %{cuda_gcc_support} evaluate to false if we don't have cuda?

cuda_gcc_support is when we have cuda i.e. we can run but we do not have a working cuda/gcc combination

and

cmsdist/rpm-preamble.file

Lines 259 to 270 in 6a36618

%if 0%{!?without_cuda:1}

%define cuda_gcc_support $(\

if [ ! -f %{_tmppath}/cache/cuda_gcc_supported.txt ] ; then \

touch %{_builddir}/cuda_gcc_supported.cu &&\

echo true > %{_builddir}/cuda_gcc_supported.txt &&\

(nvcc -dc %{_builddir}/cuda_gcc_supported.cu -o %{_builddir}/cuda_gcc_supported.cu.o 2>&1 || echo 'unsupported GCC version') > %{_builddir}/cuda_gcc_supported.log &&\

if [ $(grep -i 'unsupported ' %{_builddir}/cuda_gcc_supported.log | wc -l) -gt 0 ] ; then echo false > %{_builddir}/cuda_gcc_supported.txt ; fi;\

mv %{_builddir}/cuda_gcc_supported.txt %{_tmppath}/cache/cuda_gcc_supported.txt ;\

fi; cat %{_tmppath}/cache/cuda_gcc_supported.txt)

%else

%define cuda_gcc_support false

%endif

chage is going to force set cuda_gcc_support to false is sw stack is build without cuda

Thanks for the explanation. Will this work in a hypotetical situation where we have CUDA support, but don't build with CUDA (i.e. won't a line USE_CUDA=, where nothing is assigned to USE_CUDA, break anything?)

in the case it will work as USE_CUDA=%{!?without_cuda:ON} will never be executed when we build without_cuda

this is true that in this case USE_CUDA=%{!?without_cuda:ON} change is not needed as for without_cuda this blockof code is not executed

cmsbuild · 2024-06-03T12:35:08Z

Pull request #9218 was updated.

smuzaffar · 2024-06-03T13:48:16Z

please test

cmsbuild · 2024-06-03T22:48:37Z

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-65bebd/39674/summary.html
COMMIT: 2dfa825
CMSSW: CMSSW_14_1_X_2024-06-03-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/9218/39674/install.sh to create a dev area with all the needed externals and cmssw changes.

Build

I found compilation error when building:

/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++17 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v2 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -DEIGEN_DONT_PARALLELIZE -DEIGEN_MAX_ALIGN_BYTES=64 -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -shared -Wl,-E    -Wl,-z,defs     tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/src/HeterogeneousCoreSonicTriton/TritonClient.cc.o tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/src/HeterogeneousCoreSonicTriton/TritonData.cc.o tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/src/HeterogeneousCoreSonicTriton/TritonException.cc.o tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/src/HeterogeneousCoreSonicTriton/TritonMemResource.cc.o tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/src/HeterogeneousCoreSonicTriton/TritonService.cc.o tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/src/HeterogeneousCoreSonicTriton/triton_utils.cc.o -o tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/src/HeterogeneousCoreSonicTriton/libHeterogeneousCoreSonicTriton.so -Wl,-E -Wl,--hash-style=gnu -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_14_1_X_2024-06-03-1100/biglib/el8_amd64_gcc12 -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_14_1_X_2024-06-03-1100/lib/el8_amd64_gcc12 -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_14_1_X_2024-06-03-1100/external/el8_amd64_gcc12/lib -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02840/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-06-03-1100/biglib/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02840/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-06-03-1100/lib/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02840/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-06-02-0000/lib/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02840/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-06-02-0000/biglib/el8_amd64_gcc12 -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cuda/12.4.1-fc5cb0e72dba64b6abbf00089f3a044c/lib64/stubs -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_14_1_X_2024-06-03-1100/static/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02840/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-06-03-1100/static/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02840/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-06-02-0000/static/el8_amd64_gcc12 -lHeterogeneousCoreSonicCore -lFWCoreFramework -lFWCoreCommon -lFWCoreServiceRegistry -lDataFormatsCommon -lFWCoreParameterSet -lHeterogeneousCoreCUDAUtilities -lFWCoreMessageLogger -lDataFormatsProvenance -lFWCorePluginManager -lFWCoreReflection -lFWCoreConcurrency -lFWCoreUtilities -lFWCoreVersion -lTree -lNet -lThread -lMathCore -lRIO -lgrpcclient -ltritoncommonmodelconfig -lboost_program_options -lgrpc -lgrpc++ -lgrpc++_reflection -lCore -lboost_thread -lboost_date_time -lpcre -lbz2 -lcudart -lcudadevrt -lnvToolsExt -lnvidia-ml -luuid -lprotobuf -ltbb -llzma -lz -labsl_bad_any_cast_impl -labsl_bad_optional_access -labsl_bad_variant_access -labsl_base -labsl_city -labsl_civil_time -labsl_cord -labsl_debugging_internal -labsl_demangle_internal -labsl_examine_stack -labsl_exponential_biased -labsl_failure_signal_handler -labsl_flags_commandlineflag_internal -labsl_flags_commandlineflag -labsl_flags_config -labsl_flags_internal -labsl_flags_marshalling -labsl_flags_parse -labsl_flags_private_handle_accessor -labsl_flags_program_name -labsl_flags_reflection -labsl_flags -labsl_flags_usage_internal -labsl_flags_usage -labsl_graphcycles_internal -labsl_hash -labsl_hashtablez_sampler -labsl_int128 -labsl_leak_check -labsl_log_severity -labsl_malloc_internal -labsl_periodic_sampler -labsl_random_distributions -labsl_random_internal_distribution_test_util -labsl_random_internal_platform -labsl_random_internal_pool_urbg -labsl_random_internal_randen_hwaes_impl -labsl_random_internal_randen_hwaes -labsl_random_internal_randen_slow -labsl_random_internal_randen -labsl_random_internal_seed_material -labsl_random_seed_gen_exception -labsl_random_seed_sequences -labsl_raw_hash_set -labsl_raw_logging_internal -labsl_scoped_set_env -labsl_spinlock_wait -labsl_stacktrace -labsl_statusor -labsl_status -labsl_strerror -labsl_str_format_internal -labsl_strings_internal -labsl_strings -labsl_symbolize -labsl_synchronization -labsl_throw_delegate -labsl_time -labsl_time_zone -lcares -lcuda -lfmt -lcms-md5 -lssl -lcrypto -lre2 -lcrypt -ldl -lrt -lstdc++fs -ltinyxml2
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/src/HeterogeneousCoreSonicTriton/ccOWVrh0.ltrans2.ltrans.o: in function `TritonGpuShmResource::TritonGpuShmResource(TritonData*, std::__cxx11::basic_string, std::allocator > const&, unsigned long)':
:(.text+0x6480): undefined reference to `triton::client::InferenceServerGrpcClient::RegisterCudaSharedMemory(std::__cxx11::basic_string, std::allocator > const&, cudaIpcMemHandle_st const&, unsigned long, unsigned long, std::map, std::allocator >, std::__cxx11::basic_string, std::allocator >, std::less, std::allocator > >, std::allocator, std::allocator > const, std::__cxx11::basic_string, std::allocator > > > > const&)'
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/src/HeterogeneousCoreSonicTriton/ccOWVrh0.ltrans2.ltrans.o: in function `TritonGpuShmResource::TritonGpuShmResource(TritonData*, std::__cxx11::basic_string, std::allocator > const&, unsigned long)':
:(.text+0x6d40): undefined reference to `triton::client::InferenceServerGrpcClient::RegisterCudaSharedMemory(std::__cxx11::basic_string, std::allocator > const&, cudaIpcMemHandle_st const&, unsigned long, unsigned long, std::map, std::allocator >, std::__cxx11::basic_string, std::allocator >, std::less, std::allocator > >, std::allocator, std::allocator > const, std::__cxx11::basic_string, std::allocator > > > > const&)'
collect2: error: ld returned 1 exit status
gmake: *** [tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/src/HeterogeneousCoreSonicTriton/libHeterogeneousCoreSonicTriton.so] Error 1
Leaving library rule at HeterogeneousCore/SonicTriton
Entering library rule at src/HeterogeneousCore/SonicTriton/plugins
>> Compiling edm plugin src/HeterogeneousCore/SonicTriton/plugins/TritonService.cc
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -c -DGNU_GCC -D_GNU_SOURCE -DTBB_USE_GLIBCXX_VERSION=120301 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DCMSSW_GIT_HASH='CMSSW_14_1_X_2024-06-03-1100' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_14_1_X_2024-06-03-1100' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02840/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-06-03-1100/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/triton-inference-client/2.25.0-54d5d07371e127286e895420082bfd66/include -isystem/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/grpc/1.35.0-09ace01091c009f51007abf7a6565f16/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/pcre/8.43-e34796d17981e9b6d174328c69446455/include -isystem/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/boost/1.80.0-a1544032d9d65904ac2112b6d35bba55/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/bz2lib/1.0.6-d065ccd79984efc6d4660f410e4c81de/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cuda/12.4.1-fc5cb0e72dba64b6abbf00089f3a044c/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/libuuid/2.34-27ce4c3579b5b1de2808ea9c4cd8ed29/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/protobuf/3.21.9-437f2f0c4a1cda63055784b1b4a72f71/include -isystem/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/lcg/root/6.30.07-452c1c2d5c71d378bff01d824ff1f3c4/include -isystem/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/tbb/v2021.9.0-1dc6d66b4b014f4ae4733b04914ce7d4/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/xz/5.2.5-6f3f49b07db84e10c9be594a1176c114/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/zlib/1.2.11-1a082fc322b0051b504cc023f21df178/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/abseil-cpp/20220623.1-237da70ecfddbd80ec9c94531c1cbba7/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/c-ares/1.15.0-bf0e2c49330266ca228085a1f6d7a564/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/eigen/3bb6a48d8c171cf20b5f8e48bfb4e424fbd4f79e-3ca740c03e68b1a067f3ed0679234a78/include/eigen3 -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/fmt/8.0.1-258b4791803c34b7e98cf43693e54d87/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/md5/1.0.0-5b594b264e04ae51e893b1d69a797ec6/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/re2/2021-06-01-52c3298079c0345449c4fba721c307db/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/tinyxml2/6.2.0-88fe0ec301baf763fa3c485e5b67ed91/include -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++17 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v2 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -DEIGEN_DONT_PARALLELIZE -DEIGEN_MAX_ALIGN_BYTES=64 -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/plugins/HeterogenousCoreSonicTriton_plugins/TritonService.cc.d src/HeterogeneousCore/SonicTriton/plugins/TritonService.cc -o tmp/el8_amd64_gcc12/src/HeterogeneousCore/SonicTriton/plugins/HeterogenousCoreSonicTriton_plugins/TritonService.cc.o

smuzaffar · 2024-06-04T04:26:44Z

please test

smuzaffar · 2024-06-04T04:31:10Z

please test

cmsbuild · 2024-06-04T04:31:24Z

Pull request #9218 was updated.

cmsbuild · 2024-06-05T14:52:59Z

Pull request #9218 was updated.

smuzaffar · 2024-06-05T14:54:07Z

please test

cmsbuild · 2024-06-05T14:54:23Z

Pull request #9218 was updated.

cmsbuild · 2024-06-06T02:06:47Z

-1

Failed Tests: UnitTests RelVals RelVals-GPU AddOn GpuUnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-65bebd/39719/summary.html
COMMIT: 6f832d2
CMSSW: CMSSW_14_1_X_2024-06-05-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/9218/39719/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-65bebd/39719/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-65bebd/39719/git-merge-result

Unit Tests

I found 2 errors in the following unit tests:

---> test TestDQMOnlineClient-visualization_secondInstance had ERRORS
---> test RecoPPSLocalNewT2 had ERRORS

RelVals

RelVals-GPU

AddOn Tests

read312RV

----- Begin Fatal Exception 06-Jun-2024 03:12:38 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named /data/cmsbld/jenkins/workspace/ib-run-pr-addon/CMSSW_14_1_X_2024-06-05-1100/src/Utilities/ReleaseScripts/scripts/read312RV_cfg.py
Exception Message:
 unknown python problem occurred.
ModuleNotFoundError: No module named 'past'

At:
  /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/9218/39719/CMSSW_14_1_X_2024-06-05-1100/src/FWCore/ParameterSet/python/Types.py(6): <module>
  <frozen importlib._bootstrap>(228): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(850): exec_module
  <frozen importlib._bootstrap>(695): _load_unlocked
  <frozen importlib._bootstrap>(986): _find_and_load_unlocked
  <frozen importlib._bootstrap>(1007): _find_and_load
  /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/9218/39719/CMSSW_14_1_X_2024-06-05-1100/src/FWCore/ParameterSet/python/Config.py(16): <module>
  <frozen importlib._bootstrap>(228): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(850): exec_module
  <frozen importlib._bootstrap>(695): _load_unlocked
  <frozen importlib._bootstrap>(986): _find_and_load_unlocked
  <frozen importlib._bootstrap>(1007): _find_and_load
  /data/cmsbld/jenkins/workspace/ib-run-pr-addon/CMSSW_14_1_X_2024-06-05-1100/src/Utilities/ReleaseScripts/scripts/read312RV_cfg.py(2): <module>

----- End Fatal Exception -------------------------------------------------

fastsim

[fastsim:1] cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi  --conditions auto:run1_mc --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - elapsed time: 0 sec (ended on Thu Jun  6 03:12:41 2024) - exit: 256

fastsim1

[fastsim1:1] cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - elapsed time: 0 sec (ended on Thu Jun  6 03:12:43 2024) - exit: 256

Expand to see more addon errors ...

GPU Unit Tests

I found 5 errors in the following unit tests:

---> test testAlpakaBackendFilterCudaAsync had ERRORS
---> test testHeterogeneousCoreAlpakaTestModulesCUDA had ERRORS
---> test testHeterogeneousCoreAlpakaTestWriteReadCudaAsync had ERRORS
and more ...

smuzaffar · 2024-06-06T09:14:42Z

please test

cmsbuild · 2024-06-06T15:38:43Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-65bebd/39745/summary.html
COMMIT: 6f832d2
CMSSW: CMSSW_14_1_X_2024-06-05-2300/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/9218/39745/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test RecoPPSLocalNewT2 had ERRORS

Comparison Summary

Summary:

You potentially removed 160 lines from the logs
Reco comparison results: 9 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3338974
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3338948
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 202 log files, 165 edm output root files, 48 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

You potentially removed 8 lines from the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 3
DQMHistoTests: Total histograms compared: 39740
DQMHistoTests: Total failures: 23
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 39717
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 2 files compared)
Checked 8 log files, 10 edm output root files, 3 DQM output files
TriggerResults: no differences found

smuzaffar · 2024-06-06T19:02:06Z

+externals

cmsbuild · 2024-06-06T19:02:27Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_14_1_X/master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @rappoccio, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)
Notice This PR was tested with additional Pull Request(s), please also merge them if necessary: cms-sw/pkgtools#209

smuzaffar added 2 commits June 3, 2024 09:09

Changes needed for building cms software stack without cuda/rocm

8c74431

added missing tool files

6a36618

cmsbuild added tests-pending externals-pending pending-signatures orp-pending labels Jun 3, 2024

cmsbuild added tests-started requires-external and removed tests-pending labels Jun 3, 2024

iarspider reviewed Jun 3, 2024

View reviewed changes

Update java-env.spec

2dfa825

cmsbuild added tests-pending and removed tests-started labels Jun 3, 2024

cmsbuild added tests-started tests-rejected and removed tests-pending tests-started labels Jun 3, 2024

cmsbuild added tests-started and removed tests-rejected labels Jun 4, 2024

typo: fix triton-inference-client GPU support flag

90ca1eb

cmsbuild added tests-rejected and removed tests-started labels Jun 4, 2024

typo

b0fa63f

cmsbuild removed the tests-rejected label Jun 5, 2024

cmsbuild added the tests-pending label Jun 5, 2024

added back NVPTX

6f832d2

cmsbuild added tests-started and removed tests-pending labels Jun 5, 2024

cmsbuild added tests-rejected and removed tests-started labels Jun 6, 2024

cmsbuild added tests-started and removed tests-rejected labels Jun 6, 2024

cmsbuild added tests-rejected and removed tests-started labels Jun 6, 2024

smuzaffar merged commit 741a7f8 into IB/CMSSW_14_1_X/master Jun 6, 2024
12 of 13 checks passed

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Jun 6, 2024

cmsbuild mentioned this pull request Jun 6, 2024

Support for building software stack with options packages e.g. cuda/rocm cms-sw/pkgtools#209

Merged

smuzaffar deleted the cleanup-tools-build-without branch June 6, 2024 20:05

smuzaffar mentioned this pull request Jul 17, 2024

Fix for multi-arch: copy vectorized_packages.txt in to tool-conf #9304

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

scram tool clean; support for building stack without an option tool #9218

scram tool clean; support for building stack without an option tool #9218

smuzaffar commented Jun 3, 2024 •

edited

Loading

smuzaffar commented Jun 3, 2024 •

edited

Loading

cmsbuild commented Jun 3, 2024

cmsbuild commented Jun 3, 2024 •

edited

Loading

smuzaffar commented Jun 3, 2024

iarspider Jun 3, 2024

smuzaffar Jun 3, 2024

smuzaffar Jun 3, 2024

iarspider Jun 3, 2024 •

edited

Loading

smuzaffar Jun 3, 2024

smuzaffar Jun 3, 2024

cmsbuild commented Jun 3, 2024

smuzaffar commented Jun 3, 2024

cmsbuild commented Jun 3, 2024

smuzaffar commented Jun 4, 2024

smuzaffar commented Jun 4, 2024

cmsbuild commented Jun 4, 2024

cmsbuild commented Jun 5, 2024

smuzaffar commented Jun 5, 2024

cmsbuild commented Jun 5, 2024

cmsbuild commented Jun 6, 2024

smuzaffar commented Jun 6, 2024

cmsbuild commented Jun 6, 2024

smuzaffar commented Jun 6, 2024

cmsbuild commented Jun 6, 2024

	%if 0%{!?without_cuda:1}
	%define cuda_gcc_support $(\
	if [ ! -f %{_tmppath}/cache/cuda_gcc_supported.txt ] ; then \
	touch %{_builddir}/cuda_gcc_supported.cu &&\
	echo true > %{_builddir}/cuda_gcc_supported.txt &&\
	(nvcc -dc %{_builddir}/cuda_gcc_supported.cu -o %{_builddir}/cuda_gcc_supported.cu.o 2>&1 \|\| echo 'unsupported GCC version') > %{_builddir}/cuda_gcc_supported.log &&\
	if [ $(grep -i 'unsupported ' %{_builddir}/cuda_gcc_supported.log \| wc -l) -gt 0 ] ; then echo false > %{_builddir}/cuda_gcc_supported.txt ; fi;\
	mv %{_builddir}/cuda_gcc_supported.txt %{_tmppath}/cache/cuda_gcc_supported.txt ;\
	fi; cat %{_tmppath}/cache/cuda_gcc_supported.txt)
	%else
	%define cuda_gcc_support false
	%endif

scram tool clean; support for building stack without an option tool #9218

scram tool clean; support for building stack without an option tool #9218

Conversation

smuzaffar commented Jun 3, 2024 • edited Loading

smuzaffar commented Jun 3, 2024 • edited Loading

cmsbuild commented Jun 3, 2024

cmsbuild commented Jun 3, 2024 • edited Loading

smuzaffar commented Jun 3, 2024

iarspider Jun 3, 2024

Choose a reason for hiding this comment

smuzaffar Jun 3, 2024

Choose a reason for hiding this comment

smuzaffar Jun 3, 2024

Choose a reason for hiding this comment

iarspider Jun 3, 2024 • edited Loading

Choose a reason for hiding this comment

smuzaffar Jun 3, 2024

Choose a reason for hiding this comment

smuzaffar Jun 3, 2024

Choose a reason for hiding this comment

cmsbuild commented Jun 3, 2024

smuzaffar commented Jun 3, 2024

cmsbuild commented Jun 3, 2024

Build

smuzaffar commented Jun 4, 2024

smuzaffar commented Jun 4, 2024

cmsbuild commented Jun 4, 2024

cmsbuild commented Jun 5, 2024

smuzaffar commented Jun 5, 2024

cmsbuild commented Jun 5, 2024

cmsbuild commented Jun 6, 2024

Unit Tests

RelVals

RelVals-GPU

AddOn Tests

GPU Unit Tests

smuzaffar commented Jun 6, 2024

cmsbuild commented Jun 6, 2024

Unit Tests

Comparison Summary

GPU Comparison Summary

smuzaffar commented Jun 6, 2024

cmsbuild commented Jun 6, 2024

smuzaffar commented Jun 3, 2024 •

edited

Loading

smuzaffar commented Jun 3, 2024 •

edited

Loading

cmsbuild commented Jun 3, 2024 •

edited

Loading

iarspider Jun 3, 2024 •

edited

Loading