Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault in dc_mlx5/test_uct_stats.atomic_swap64/0 #1542

Closed
brminich opened this issue May 23, 2017 · 1 comment
Closed

Segmentation fault in dc_mlx5/test_uct_stats.atomic_swap64/0 #1542

brminich opened this issue May 23, 2017 · 1 comment
Assignees
Labels
Milestone

Comments

@brminich
Copy link
Contributor

brminich commented May 23, 2017

Found in:
http://hpc-master.lab.mtl.com:8080/job/hpc-ucx-pr/label=hpc-test-node,worker=0/3658/console

$gdb /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/build-test/test/gtest/gtest
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/build-test/test/gtest/gtest...done.
(gdb) target remote|vgdb
Remote debugging using |vgdb
relaying data between gdb and process 14169
Reading symbols from /usr/lib64/valgrind/vgpreload_core-amd64-linux.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/valgrind/vgpreload_core-amd64-linux.so
Reading symbols from /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so
Reading symbols from /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/build-test/src/ucs/.libs/libucs.so.0...done.
Loaded symbols for /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/build-test/src/ucs/.libs/libucs.so.0
Reading symbols from /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/build-test/src/uct/.libs/libuct.so.0...done.
Loaded symbols for /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/build-test/src/uct/.libs/libuct.so.0
Reading symbols from /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/build-test/src/ucm/.libs/libucm.so.0...done.
Loaded symbols for /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/build-test/src/ucm/.libs/libucm.so.0
Reading symbols from /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/build-test/src/ucp/.libs/libucp.so.0...done.
Loaded symbols for /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/build-test/src/ucp/.libs/libucp.so.0
Reading symbols from /usr/lib64/mlnx_ofed/valgrind/libibverbs.so.1...Reading symbols from /usr/lib64/mlnx_ofed/valgrind/libibverbs.so.1...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/mlnx_ofed/valgrind/libibverbs.so.1
Reading symbols from /usr/lib64/libnuma.so.1...Reading symbols from /usr/lib64/libnuma.so.1...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libnuma.so.1
Reading symbols from /usr/lib64/mlnx_ofed/valgrind/libibcm.so.1...Reading symbols from /usr/lib64/mlnx_ofed/valgrind/libibcm.so.1...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/mlnx_ofed/valgrind/libibcm.so.1
Reading symbols from /usr/lib64/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libpthread.so.0
Reading symbols from /usr/lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/librt.so.1
Reading symbols from /usr/lib64/libz.so.1...Reading symbols from /usr/lib64/libz.so.1...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libz.so.1
Reading symbols from /usr/lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libdl.so.2
Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libstdc++.so.6
Reading symbols from /usr/lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libm.so.6
Reading symbols from /usr/lib64/libgomp.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libgomp.so.1
Reading symbols from /usr/lib64/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libgcc_s.so.1
Reading symbols from /usr/lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libc.so.6
Reading symbols from /usr/lib64/libnl.so.1...Reading symbols from /usr/lib/debug/usr/lib64/libnl.so.1.1.4.debug...done.
done.
Loaded symbols for /usr/lib64/libnl.so.1
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /usr/lib64/mlnx_ofed/valgrind/libmlx5-rdmav2.so...Reading symbols from /usr/lib64/mlnx_ofed/valgrind/libmlx5-rdmav2.so...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/mlnx_ofed/valgrind/libmlx5-rdmav2.so
Reading symbols from /usr/lib64/mlnx_ofed/valgrind/libmlx4-rdmav2.so...Reading symbols from /usr/lib64/mlnx_ofed/valgrind/libmlx4-rdmav2.so...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/mlnx_ofed/valgrind/libmlx4-rdmav2.so
0x0000000005eb6a2d in pause () from /usr/lib64/libpthread.so.0
Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.el7_2.6.x86_64 libgcc-4.8.5-11.el7.x86_64 libgomp-4.8.5-11.el7.x86_64 libibcm-1.0.5mlnx2-OFED.3.4.0.0.4.34100.x86_64 libibverbs-1.2.1mlnx1-OFED.3.4.2.1.4.34219.x86_64 libmlx4-1.2.1mlnx1-OFED.3.4.0.0.4.34219.x86_64 libmlx5-1.2.1mlnx1-OFED.3.4.2.1.9.34219.x86_64 libstdc++-4.8.5-11.el7.x86_64 numactl-libs-2.0.9-6.el7_2.x86_64 valgrind-3.10.0-16.el7.x86_64 zlib-1.2.7-15.el7.x86_64
(gdb) bt
#0  0x0000000005eb6a2d in pause () from /usr/lib64/libpthread.so.0
#1  0x0000000004e7da55 in ucs_debug_freeze () at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../src/ucs/debug/debug.c:735
#2  0x0000000004e7f685 in ucs_error_freeze (message=0x40453c0 "Caught signal 11 (Segmentation fault: invalid permissions for mapped object at address (nil))", error_type=0x4f277a8 "invalid permissions for mapped object")
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../src/ucs/debug/debug.c:854
#3  ucs_handle_error (error_type=error_type@entry=0x4f277a8 "invalid permissions for mapped object", message=message@entry=0x4f271a3 "Caught signal %d (%s: %s%s)")
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../src/ucs/debug/debug.c:1018
#4  0x0000000004e7fa47 in ucs_debug_handle_error_signal (signo=signo@entry=11, cause=0x4f277a8 "invalid permissions for mapped object", fmt=fmt@entry=0x4f2746b " at address %p")
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../src/ucs/debug/debug.c:958
#5  0x0000000004e801cb in ucs_error_signal_handler (signo=11, info=0x4045c70, context=<optimized out>) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../src/ucs/debug/debug.c:981
#6  <signal handler called>
#7  0x0000000000000000 in ?? ()
#8  0x0000000004286380 in ?? ()
#9  0x0000000000521790 in test_uct_stats::cleanup (this=0x4286380) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/uct/test_stats.cc:42
#10 0x000000000046878a in ucs::test_base::TearDownProxy (this=0x4286398) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/test.cc:185
#11 0x00000000004601c3 in HandleSehExceptionsInMethodIfSupported<testing::Test, void> (location=0x72ab3a "TearDown()", method=<optimized out>, object=<optimized out>)
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest-all.cc:3562
#12 testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=object@entry=0x4286380, method=<optimized out>, location=location@entry=0x72ab3a "TearDown()")
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest-all.cc:3598
#13 0x00000000004541bc in testing::Test::Run (this=this@entry=0x4286380) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest-all.cc:3643
#14 0x00000000004542bc in testing::TestInfo::Run (this=0x8e1ac60) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest-all.cc:3812
#15 0x000000000045441f in testing::TestCase::Run (this=0x8ba4c50) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest-all.cc:3930
#16 0x0000000000458f37 in testing::internal::UnitTestImpl::RunAllTests (this=this@entry=0x7766c60) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest-all.cc:5802
#17 0x000000000045923b in RunAllTests (this=0x7766c60) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest-all.cc:5719
#18 HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (location=<optimized out>, method=<optimized out>, object=0x7766c60)
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest-all.cc:3562
#19 HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (location=0x72c010 "auxiliary test code (environments or event listeners)", 
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x459180 <testing::internal::UnitTestImpl::RunAllTests()>, object=0x7766c60)
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest-all.cc:3598
#20 testing::UnitTest::Run (this=<optimized out>) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest-all.cc:5416
#21 0x000000000040e9f2 in RUN_ALL_TESTS () at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/gtest.h:20059
#22 main (argc=1, argv=<optimized out>) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/common/main.cc:79
(gdb) f 9
#9  0x0000000000521790 in test_uct_stats::cleanup (this=0x4286380) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/uct/test_stats.cc:42
42                   uct_p2p_test::cleanup();
(gdb) l
37               }
38           
39               virtual void cleanup() {
40                   delete lbuf;
41                   delete rbuf;
42                   uct_p2p_test::cleanup();
43                   stats_restore();
44               }
45           
46               uct_base_ep_t *uct_ep(const entity &e)
(gdb) disas
Dump of assembler code for function test_uct_stats::cleanup():
   0x0000000000521760 <+0>:      push   %rbx
   0x0000000000521761 <+1>:      mov    %rdi,%rbx
   0x0000000000521764 <+4>:      mov    0xd0(%rdi),%rdi
   0x000000000052176b <+11>:    test   %rdi,%rdi
   0x000000000052176e <+14>:    je     0x521776 <test_uct_stats::cleanup()+22>
   0x0000000000521770 <+16>:    mov    (%rdi),%rax
   0x0000000000521773 <+19>:    callq  *0x8(%rax)
   0x0000000000521776 <+22>:    mov    0xd8(%rbx),%rdi
   0x000000000052177d <+29>:    test   %rdi,%rdi
   0x0000000000521780 <+32>:    je     0x521788 <test_uct_stats::cleanup()+40>
   0x0000000000521782 <+34>:    mov    (%rdi),%rax
   0x0000000000521785 <+37>:    callq  *0x8(%rax)
   0x0000000000521788 <+40>:    mov    %rbx,%rdi
   0x000000000052178b <+43>:    callq  0x50eed0 <uct_p2p_test::cleanup()>
=> 0x0000000000521790 <+48>: mov    %rbx,%rdi
   0x0000000000521793 <+51>:    pop    %rbx
   0x0000000000521794 <+52>:    jmpq   0x5123d0 <uct_test::stats_restore()>
End of assembler dump.
(gdb) l uct_p2p_test::cleanup
85           
86               /* Allocate completion handle and set the callback */
87               m_completion_count = 0;
88           }
89           
90           void uct_p2p_test::cleanup() {
91           
92               flush();
93               uct_test::cleanup();
94           }
(gdb) disas uct_p2p_test::cleanup
`<unnamed type>' is not defined as an aggregate type.
(gdb) disas & uct_p2p_test::cleanup
`<unnamed type>' is not defined as an aggregate type.
(gdb) disas 0x50eed0
Dump of assembler code for function uct_p2p_test::cleanup():
   0x000000000050eed0 <+0>:     push   %rbx
   0x000000000050eed1 <+1>:     mov    %rdi,%rbx
   0x000000000050eed4 <+4>:     callq  0x513260 <uct_test::flush() const>
   0x000000000050eed9 <+9>:     mov    %rbx,%rdi
   0x000000000050eedc <+12>:   pop    %rbx
   0x000000000050eedd <+13>:   jmpq   0x511a20 <uct_test::cleanup()>
End of assembler dump.
(gdb) disas 0x511a20
Dump of assembler code for function uct_test::cleanup():
   0x0000000000511a20 <+0>:      push   %r15
   0x0000000000511a22 <+2>:      push   %r14
   0x0000000000511a24 <+4>:      push   %r13
   0x0000000000511a26 <+6>:      push   %r12
   0x0000000000511a28 <+8>:      push   %rbp
   0x0000000000511a29 <+9>:      mov    %rdi,%rbp
   0x0000000000511a2c <+12>:    push   %rbx
   0x0000000000511a2d <+13>:    sub    $0x8,%rsp
   0x0000000000511a31 <+17>:    mov    0x78(%rdi),%r15
   0x0000000000511a35 <+21>:    mov    0x80(%rdi),%rdx
   0x0000000000511a3c <+28>:    cmp    %rdx,%r15
   0x0000000000511a3f <+31>:     je     0x511bf3 <uct_test::cleanup()+467>
   0x0000000000511a45 <+37>:    movabs $0xaaaaaaaaaaaaaaab,%r13
   0x0000000000511a4f <+47>:     nop
   0x0000000000511a50 <+48>:    mov    (%r15),%r14
   0x0000000000511a53 <+51>:    mov    0x158(%r14),%rsi
   0x0000000000511a5a <+58>:    mov    0x150(%r14),%rax
   0x0000000000511a61 <+65>:    mov    %rsi,%rcx
   0x0000000000511a64 <+68>:    sub    %rax,%rcx
   0x0000000000511a67 <+71>:    sar    $0x3,%rcx
   0x0000000000511a6b <+75>:    imul   %r13,%rcx
   0x0000000000511a6f <+79>:     test   %rcx,%rcx
   0x0000000000511a72 <+82>:    je     0x511ad1 <uct_test::cleanup()+177>
   0x0000000000511a74 <+84>:    xor    %ebx,%ebx
   0x0000000000511a76 <+86>:    xor    %edx,%edx
   0x0000000000511a78 <+88>:    jmp    0x511a83 <uct_test::cleanup()+99>
   0x0000000000511a7a <+90>:    nopw   0x0(%rax,%rax,1)
   0x0000000000511a80 <+96>:    mov    %rcx,%rax
   0x0000000000511a83 <+99>:    lea    (%rdx,%rdx,2),%rdx
   0x0000000000511a87 <+103>:  mov    %rax,%rcx
   0x0000000000511a8a <+106>:  lea    (%rax,%rdx,8),%r12
   0x0000000000511a8e <+110>: cmpb   $0x0,(%r12)
   0x0000000000511a93 <+115>:  je     0x511ab2 <uct_test::cleanup()+146>
   0x0000000000511a95 <+117>:  mov    0x8(%r12),%rdi
   0x0000000000511a9a <+122>:  callq  *0x10(%r12)
   0x0000000000511a9f <+127>:  mov    0x150(%r14),%rcx
   0x0000000000511aa6 <+134>:  mov    0x158(%r14),%rsi
   0x0000000000511aad <+141>: movb   $0x0,(%r12)
   0x0000000000511ab2 <+146>: mov    %rsi,%rax
   0x0000000000511ab5 <+149>: add    $0x1,%ebx
   0x0000000000511ab8 <+152>: sub    %rcx,%rax
   0x0000000000511abb <+155>: mov    %ebx,%edx
   0x0000000000511abd <+157>: sar    $0x3,%rax
   0x0000000000511ac1 <+161>:  imul   %r13,%rax
---Type <return> to continue, or q <return> to quit---
   0x0000000000511ac5 <+165>:  cmp    %rax,%rdx
   0x0000000000511ac8 <+168>:  jb     0x511a80 <uct_test::cleanup()+96>
   0x0000000000511aca <+170>:  mov    0x80(%rbp),%rdx
   0x0000000000511ad1 <+177>: add    $0x8,%r15
   0x0000000000511ad5 <+181>: cmp    %rdx,%r15
   0x0000000000511ad8 <+184>: jne    0x511a50 <uct_test::cleanup()+48>
   0x0000000000511ade <+190>: mov    0x78(%rbp),%rax
   0x0000000000511ae2 <+194>: cmp    %rdx,%rax
   0x0000000000511ae5 <+197>: je     0x511bb8 <uct_test::cleanup()+408>
   0x0000000000511aeb <+203>: mov    -0x8(%rdx),%r13
   0x0000000000511aef <+207>:  sub    $0x8,%rdx
   0x0000000000511af3 <+211>:  mov    %rdx,0x80(%rbp)
   0x0000000000511afa <+218>:  test   %r13,%r13
   0x0000000000511afd <+221>:  je     0x511ae2 <uct_test::cleanup()+194>
   0x0000000000511aff <+223>:   mov    0x158(%r13),%r12
   0x0000000000511b06 <+230>: mov    0x150(%r13),%rbx
   0x0000000000511b0d <+237>: cmp    %rbx,%r12
   0x0000000000511b10 <+240>: jne    0x511b29 <uct_test::cleanup()+265>
   0x0000000000511b12 <+242>: jmpq   0x511beb <uct_test::cleanup()+459>
   0x0000000000511b17 <+247>: nopw   0x0(%rax,%rax,1)
   0x0000000000511b20 <+256>: add    $0x18,%rbx
   0x0000000000511b24 <+260>: cmp    %rbx,%r12
   0x0000000000511b27 <+263>: je     0x511b41 <uct_test::cleanup()+289>
   0x0000000000511b29 <+265>: cmpb   $0x0,(%rbx)
   0x0000000000511b2c <+268>:  je     0x511b20 <uct_test::cleanup()+256>
   0x0000000000511b2e <+270>: mov    0x8(%rbx),%rdi
   0x0000000000511b32 <+274>: callq  *0x10(%rbx)
   0x0000000000511b35 <+277>: movb   $0x0,(%rbx)
   0x0000000000511b38 <+280>: add    $0x18,%rbx
   0x0000000000511b3c <+284>:  cmp    %rbx,%r12
   0x0000000000511b3f <+287>:  jne    0x511b29 <uct_test::cleanup()+265>
   0x0000000000511b41 <+289>: mov    0x150(%r13),%rdi
   0x0000000000511b48 <+296>: test   %rdi,%rdi
   0x0000000000511b4b <+299>: je     0x511b52 <uct_test::cleanup()+306>
   0x0000000000511b4d <+301>: callq  0x40b750 <_ZdlPv@plt>
   0x0000000000511b52 <+306>: cmpb   $0x0,0x138(%r13)
   0x0000000000511b5a <+314>: jne    0x511bd0 <uct_test::cleanup()+432>
   0x0000000000511b5c <+316>:  cmpb   $0x0,0x120(%r13)
   0x0000000000511b64 <+324>: je     0x511b7c <uct_test::cleanup()+348>
   0x0000000000511b66 <+326>: mov    0x128(%r13),%rdi
   0x0000000000511b6d <+333>: callq  *0x130(%r13)
   0x0000000000511b74 <+340>: movb   $0x0,0x120(%r13)
   0x0000000000511b7c <+348>:  lea    0xd0(%r13),%rdi
   0x0000000000511b83 <+355>: callq  0x40bc60 <ucs_async_context_cleanup@plt>
   0x0000000000511b88 <+360>: cmpb   $0x0,0x0(%r13)
---Type <return> to continue, or q <return> to quit---
   0x0000000000511b8d <+365>: je     0x511b9c <uct_test::cleanup()+380>
   0x0000000000511b8f <+367>:  mov    0x8(%r13),%rdi
   0x0000000000511b93 <+371>: callq  *0x10(%r13)
   0x0000000000511b97 <+375>: movb   $0x0,0x0(%r13)
   0x0000000000511b9c <+380>:  mov    %r13,%rdi
   0x0000000000511b9f <+383>:  callq  0x40b750 <_ZdlPv@plt>
   0x0000000000511ba4 <+388>: mov    0x78(%rbp),%rax
   0x0000000000511ba8 <+392>: mov    0x80(%rbp),%rdx
   0x0000000000511baf <+399>:  cmp    %rdx,%rax
   0x0000000000511bb2 <+402>: jne    0x511aeb <uct_test::cleanup()+203>
   0x0000000000511bb8 <+408>: add    $0x8,%rsp
   0x0000000000511bbc <+412>: pop    %rbx
   0x0000000000511bbd <+413>: pop    %rbp
   0x0000000000511bbe <+414>: pop    %r12
   0x0000000000511bc0 <+416>:  pop    %r13
   0x0000000000511bc2 <+418>:  pop    %r14
   0x0000000000511bc4 <+420>:  pop    %r15
   0x0000000000511bc6 <+422>:  retq   
   0x0000000000511bc7 <+423>:  nopw   0x0(%rax,%rax,1)
   0x0000000000511bd0 <+432>: mov    0x140(%r13),%rdi
   0x0000000000511bd7 <+439>: callq  *0x148(%r13)
   0x0000000000511bde <+446>: movb   $0x0,0x138(%r13)
   0x0000000000511be6 <+454>: jmpq   0x511b5c <uct_test::cleanup()+316>
   0x0000000000511beb <+459>: mov    %r12,%rdi
   0x0000000000511bee <+462>: jmpq   0x511b48 <uct_test::cleanup()+296>
   0x0000000000511bf3 <+467>:  mov    %r15,%rdx
   0x0000000000511bf6 <+470>:  mov    %r15,%rax
   0x0000000000511bf9 <+473>:  jmpq   0x511ae2 <uct_test::cleanup()+194>
   0x0000000000511bfe <+478>:  mov    0x150(%r13),%rdi
   0x0000000000511c05 <+485>:  mov    %rax,%rbx
   0x0000000000511c08 <+488>:  test   %rdi,%rdi
   0x0000000000511c0b <+491>:  je     0x511c7d <uct_test::cleanup()+605>
   0x0000000000511c0d <+493>:  vzeroupper 
   0x0000000000511c10 <+496>:  callq  0x40b750 <_ZdlPv@plt>
   0x0000000000511c15 <+501>:  cmpb   $0x0,0x138(%r13)
   0x0000000000511c1d <+509>:  je     0x511c35 <uct_test::cleanup()+533>
   0x0000000000511c1f <+511>:   mov    0x140(%r13),%rdi
   0x0000000000511c26 <+518>:  callq  *0x148(%r13)
   0x0000000000511c2d <+525>:  movb   $0x0,0x138(%r13)
   0x0000000000511c35 <+533>:  cmpb   $0x0,0x120(%r13)
   0x0000000000511c3d <+541>:  je     0x511c55 <uct_test::cleanup()+565>
   0x0000000000511c3f <+543>:   mov    0x128(%r13),%rdi
   0x0000000000511c46 <+550>:  callq  *0x130(%r13)
   0x0000000000511c4d <+557>:  movb   $0x0,0x120(%r13)
   0x0000000000511c55 <+565>:  lea    0xd0(%r13),%rdi
---Type <return> to continue, or q <return> to quit---
   0x0000000000511c5c <+572>:  callq  0x40bc60 <ucs_async_context_cleanup@plt>
   0x0000000000511c61 <+577>:  cmpb   $0x0,0x0(%r13)
   0x0000000000511c66 <+582>:  je     0x511c75 <uct_test::cleanup()+597>
   0x0000000000511c68 <+584>:  mov    0x8(%r13),%rdi
   0x0000000000511c6c <+588>:  callq  *0x10(%r13)
   0x0000000000511c70 <+592>:  movb   $0x0,0x0(%r13)
   0x0000000000511c75 <+597>:  mov    %rbx,%rdi
   0x0000000000511c78 <+600>:  callq  0x40c7c0 <_Unwind_Resume@plt>
   0x0000000000511c7d <+605>:  vzeroupper 
   0x0000000000511c80 <+608>:  jmp    0x511c15 <uct_test::cleanup()+501>
   0x0000000000511c82 <+610>:  mov    %rax,%rbx
   0x0000000000511c85 <+613>:  vzeroupper 
   0x0000000000511c88 <+616>:  jmp    0x511c35 <uct_test::cleanup()+533>
   0x0000000000511c8a <+618>:  mov    %rax,%rbx
   0x0000000000511c8d <+621>:  vzeroupper 
   0x0000000000511c90 <+624>:  jmp    0x511c61 <uct_test::cleanup()+577>
   0x0000000000511c92 <+626>:  mov    %rax,%rbx
   0x0000000000511c95 <+629>:  vzeroupper 
   0x0000000000511c98 <+632>:  jmp    0x511c55 <uct_test::cleanup()+565>
End of assembler dump.
(gdb) f 9
#9  0x0000000000521790 in test_uct_stats::cleanup (this=0x4286380) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr-2/label/hpc-test-node/worker/0/contrib/../test/gtest/uct/test_stats.cc:42
42                   uct_p2p_test::cleanup();
(gdb) l
37               }
38           
39               virtual void cleanup() {
40                   delete lbuf;
41                   delete rbuf;
42                   uct_p2p_test::cleanup();
43                   stats_restore();
44               }
45           
46               uct_base_ep_t *uct_ep(const entity &e)
(gdb) p m_entities
$1 = {<ucs::ptr_vector_base<uct_test::entity>> = {_vptr.ptr_vector_base = 0x737630 <vtable for ucs::ptr_vector<uct_test::entity>+16>, m_vec = std::vector of length 2, capacity 2 = {0x4277d40, 0x427cf10}}, <No data fields>}
(gdb) p m_entities.m_vec
$2 = std::vector of length 2, capacity 2 = {0x4277d40, 0x427cf10}
(gdb) p m_entities.m_vec._M_impl
$3 = {<std::allocator<uct_test::entity*>> = {<__gnu_cxx::new_allocator<uct_test::entity*>> = {<No data fields>}, <No data fields>}, _M_start = 0x427f4c0, _M_finish = 0x427f4d0, _M_end_of_storage = 0x427f4d0}
(gdb) p m_entities.m_vec._M_impl._M_start[0]
$4 = (uct_test::entity *) 0x4277d40
(gdb) p *m_entities.m_vec._M_impl._M_start[0]
$5 = {m_md = {m_initialized = true, m_value = 0x4278610, m_dtor = 0x40c620 <uct_md_close@plt>}, m_md_attr = {cap = {max_alloc = 18446744073709551615, max_reg = 18446744073709551615, flags = 30}, reg_cost = {
      overhead = 8.9999999999999999e-08, growth = 0}, component_name = "ib\000\000\000\000\000", rkey_packed_size = 16, local_cpus = {__bits = {16711935, 0 <repeats 15 times>}}}, m_async = {m_async = {{thread = {{mutex = {
              __data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}, 
            spinlock = {lock = 0, count = 0, owner = 0}}}, signal = {tid = 0, block_count = 0, pthread = 0, timer = 0x1}, poll_block = 0}, mode = UCS_ASYNC_MODE_THREAD, num_handlers = 0, missed = {length = 1024, shift = 10, 
        producer = 0, consumer = 0, queue = 0x427d1c0}, last_wakeup = 32175441071142537}}, m_worker = {m_initialized = true, m_value = 0x425b8f0, m_dtor = 0x40b700 <uct_worker_destroy@plt>}, m_iface = {m_initialized = true, 
    m_value = 0x4247d20, m_dtor = 0x40b060 <uct_iface_close@plt>}, m_eps = std::vector of length 1, capacity 1 = {{m_initialized = true, m_value = 0x4304700, m_dtor = 0x40b250 <uct_ep_destroy@plt>}}, m_iface_attr = {cap = {put = {
        max_short = 172, max_bcopy = 8192, min_zcopy = 0, max_zcopy = 1073741824, opt_zcopy_align = 512, align_mtu = 4096, max_iov = 8}, get = {max_bcopy = 8192, min_zcopy = 65, max_zcopy = 1073741824, opt_zcopy_align = 512, 
        align_mtu = 4096, max_iov = 8}, am = {max_short = 187, max_bcopy = 8191, min_zcopy = 0, max_zcopy = 8191, opt_zcopy_align = 512, align_mtu = 4096, max_hdr = 139, max_iov = 3}, tag = {recv = {min_recv = 0, max_iov = 0}, 
        eager = {max_short = 0, max_bcopy = 0, max_zcopy = 0, max_iov = 0}, rndv = {max_zcopy = 0, max_hdr = 0, max_iov = 0}}, flags = 18968739776127}, device_addr_len = 3, iface_addr_len = 4, ep_addr_len = 0, 
    overhead = 4.0000000000000001e-08, bandwidth = 6745718050.065876, latency = {overhead = 9.5999999999999991e-07, growth = 0}, priority = 20 '\024'}}
(gdb) p *m_entities.m_vec._M_impl._M_start[1]
$6 = {m_md = {m_initialized = true, m_value = 0x423d560, m_dtor = 0x40c620 <uct_md_close@plt>}, m_md_attr = {cap = {max_alloc = 18446744073709551615, max_reg = 18446744073709551615, flags = 30}, reg_cost = {
      overhead = 8.9999999999999999e-08, growth = 0}, component_name = "ib\000\000\000\000\000", rkey_packed_size = 16, local_cpus = {__bits = {16711935, 0 <repeats 15 times>}}}, m_async = {m_async = {{thread = {{mutex = {
              __data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}, 
            spinlock = {lock = 0, count = 0, owner = 0}}}, signal = {tid = 0, block_count = 0, pthread = 0, timer = 0x1}, poll_block = 0}, mode = UCS_ASYNC_MODE_THREAD, num_handlers = 0, missed = {length = 1024, shift = 10, 
        producer = 0, consumer = 0, queue = 0x423bf30}, last_wakeup = 32175441593242761}}, m_worker = {m_initialized = true, m_value = 0x4260d60, m_dtor = 0x40b700 <uct_worker_destroy@plt>}, m_iface = {m_initialized = true, 
    m_value = 0x426ca30, m_dtor = 0x40b060 <uct_iface_close@plt>}, m_eps = std::vector of length 1, capacity 1 = {{m_initialized = true, m_value = 0x4304990, m_dtor = 0x40b250 <uct_ep_destroy@plt>}}, m_iface_attr = {cap = {put = {
        max_short = 172, max_bcopy = 8192, min_zcopy = 0, max_zcopy = 1073741824, opt_zcopy_align = 512, align_mtu = 4096, max_iov = 8}, get = {max_bcopy = 8192, min_zcopy = 65, max_zcopy = 1073741824, opt_zcopy_align = 512, 
        align_mtu = 4096, max_iov = 8}, am = {max_short = 187, max_bcopy = 8191, min_zcopy = 0, max_zcopy = 8191, opt_zcopy_align = 512, align_mtu = 4096, max_hdr = 139, max_iov = 3}, tag = {recv = {min_recv = 0, max_iov = 0}, 
        eager = {max_short = 0, max_bcopy = 0, max_zcopy = 0, max_iov = 0}, rndv = {max_zcopy = 0, max_hdr = 0, max_iov = 0}}, flags = 18968739776127}, device_addr_len = 3, iface_addr_len = 4, ep_addr_len = 0, 
    overhead = 4.0000000000000001e-08, bandwidth = 6745718050.065876, latency = {overhead = 9.5999999999999991e-07, growth = 0}, priority = 20 '\024'}}
(gdb) p *m_entities.m_vec._M_impl._M_start[0]->destroy_eps
Attempt to take contents of a non-pointer value.
(gdb) p m_entities.m_vec._M_impl._M_start[0]->destroy_eps
$7 = {void (uct_test::entity * const)} 0x5125c0 <uct_test::entity::destroy_eps()>
(gdb) p m_entities.m_vec._M_impl._M_start[1]->destroy_eps
$8 = {void (uct_test::entity * const)} 0x5125c0 <uct_test::entity::destroy_eps()>
(gdb) p m_entities.m_vec._M_impl._M_start[0]->m_eps
$9 = std::vector of length 1, capacity 1 = {{m_initialized = true, m_value = 0x4304700, m_dtor = 0x40b250 <uct_ep_destroy@plt>}}
(gdb) p m_entities.m_vec._M_impl._M_start[1]->m_eps
$10 = std::vector of length 1, capacity 1 = {{m_initialized = true, m_value = 0x4304990, m_dtor = 0x40b250 <uct_ep_destroy@plt>}}
(gdb) p m_entities.m_vec._M_impl._M_start[0]->m_eps._M_impl._M_start[0]
$11 = {m_initialized = true, m_value = 0x4304700, m_dtor = 0x40b250 <uct_ep_destroy@plt>}
(gdb) p m_entities.m_vec._M_impl._M_start[0]->m_eps._M_impl._M_start[0].m_value
$12 = (uct_ep *) 0x4304700
(gdb) p m_entities.m_vec._M_impl._M_start[0]->m_eps._M_impl._M_start[0].m_value->iface
$13 = (uct_iface_h) 0x4247d20
(gdb) p m_entities.m_vec._M_impl._M_start[0]->m_eps._M_impl._M_start[0].m_value->iface.ops
$14 = {iface_close = 0x51c7370 <uct_dc_mlx5_iface_t_delete>, iface_query = 0x51c7540 <uct_dc_mlx5_iface_query>, iface_flush = 0x51bc620 <uct_dc_iface_flush>, iface_fence = 0x5199250 <uct_base_iface_fence>, iface_wakeup_open = 0x0, 
  iface_wakeup_get_fd = 0x0, iface_wakeup_arm = 0x0, iface_wakeup_wait = 0x0, iface_wakeup_signal = 0x0, iface_wakeup_close = 0x0, iface_tag_recv_zcopy = 0x0, iface_tag_recv_cancel = 0x0, ep_create = 0x0, 
  ep_create_connected = 0x51c8750 <uct_dc_mlx5_ep_t_new>, ep_destroy = 0x51c7520 <uct_dc_mlx5_ep_destroy>, ep_get_address = 0x0, ep_connect_to_ep = 0x0, iface_get_device_address = 0x519f4a0 <uct_ib_iface_get_device_address>, 
  iface_get_address = 0x51bc5d0 <uct_dc_iface_get_address>, iface_is_reachable = 0x519f500 <uct_ib_iface_is_reachable>, ep_put_short = 0x51d04c0 <uct_dc_mlx5_ep_put_short>, ep_put_bcopy = 0x51d0e10 <uct_dc_mlx5_ep_put_bcopy>, 
  ep_put_zcopy = 0x51c8c50 <uct_dc_mlx5_ep_put_zcopy>, ep_get_bcopy = 0x51cb570 <uct_dc_mlx5_ep_get_bcopy>, ep_get_zcopy = 0x51cf780 <uct_dc_mlx5_ep_get_zcopy>, ep_am_short = 0x51cdfa0 <uct_dc_mlx5_ep_am_short>, 
  ep_am_bcopy = 0x51ccab0 <uct_dc_mlx5_ep_am_bcopy>, ep_am_zcopy = 0x51c9a40 <uct_dc_mlx5_ep_am_zcopy>, ep_atomic_add64 = 0x51d4b30 <uct_dc_mlx5_ep_atomic_add64>, ep_atomic_fadd64 = 0x51cab00 <uct_dc_mlx5_ep_atomic_fadd64>, 
  ep_atomic_swap64 = 0x51d22e0 <uct_dc_mlx5_ep_atomic_swap64>, ep_atomic_cswap64 = 0x51d1870 <uct_dc_mlx5_ep_atomic_cswap64>, ep_atomic_add32 = 0x51d4b50 <uct_dc_mlx5_ep_atomic_add32>, 
  ep_atomic_fadd32 = 0x51d2da0 <uct_dc_mlx5_ep_atomic_fadd32>, ep_atomic_swap32 = 0x51cd520 <uct_dc_mlx5_ep_atomic_swap32>, ep_atomic_cswap32 = 0x51cc040 <uct_dc_mlx5_ep_atomic_cswap32>, 
  ep_pending_add = 0x51bd9b0 <uct_dc_ep_pending_add>, ep_pending_purge = 0x51bded0 <uct_dc_ep_pending_purge>, ep_flush = 0x51c7840 <uct_dc_mlx5_ep_flush>, ep_fence = 0x5199290 <uct_base_ep_fence>, ep_check = 0x0, 
  ep_tag_eager_short = 0x0, ep_tag_eager_bcopy = 0x0, ep_tag_eager_zcopy = 0x0, ep_tag_rndv_zcopy = 0x0, ep_tag_rndv_cancel = 0x0, ep_tag_rndv_request = 0x0}
(gdb) p m_entities.m_vec._M_impl._M_start[1]->m_eps._M_impl._M_start[0].m_value->iface.ops
$15 = {iface_close = 0x51c7370 <uct_dc_mlx5_iface_t_delete>, iface_query = 0x51c7540 <uct_dc_mlx5_iface_query>, iface_flush = 0x51bc620 <uct_dc_iface_flush>, iface_fence = 0x5199250 <uct_base_iface_fence>, iface_wakeup_open = 0x0, 
  iface_wakeup_get_fd = 0x0, iface_wakeup_arm = 0x0, iface_wakeup_wait = 0x0, iface_wakeup_signal = 0x0, iface_wakeup_close = 0x0, iface_tag_recv_zcopy = 0x0, iface_tag_recv_cancel = 0x0, ep_create = 0x0, 
  ep_create_connected = 0x51c8750 <uct_dc_mlx5_ep_t_new>, ep_destroy = 0x51c7520 <uct_dc_mlx5_ep_destroy>, ep_get_address = 0x0, ep_connect_to_ep = 0x0, iface_get_device_address = 0x519f4a0 <uct_ib_iface_get_device_address>, 
  iface_get_address = 0x51bc5d0 <uct_dc_iface_get_address>, iface_is_reachable = 0x519f500 <uct_ib_iface_is_reachable>, ep_put_short = 0x51d04c0 <uct_dc_mlx5_ep_put_short>, ep_put_bcopy = 0x51d0e10 <uct_dc_mlx5_ep_put_bcopy>, 
  ep_put_zcopy = 0x51c8c50 <uct_dc_mlx5_ep_put_zcopy>, ep_get_bcopy = 0x51cb570 <uct_dc_mlx5_ep_get_bcopy>, ep_get_zcopy = 0x51cf780 <uct_dc_mlx5_ep_get_zcopy>, ep_am_short = 0x51cdfa0 <uct_dc_mlx5_ep_am_short>, 
  ep_am_bcopy = 0x51ccab0 <uct_dc_mlx5_ep_am_bcopy>, ep_am_zcopy = 0x51c9a40 <uct_dc_mlx5_ep_am_zcopy>, ep_atomic_add64 = 0x51d4b30 <uct_dc_mlx5_ep_atomic_add64>, ep_atomic_fadd64 = 0x51cab00 <uct_dc_mlx5_ep_atomic_fadd64>, 
  ep_atomic_swap64 = 0x51d22e0 <uct_dc_mlx5_ep_atomic_swap64>, ep_atomic_cswap64 = 0x51d1870 <uct_dc_mlx5_ep_atomic_cswap64>, ep_atomic_add32 = 0x51d4b50 <uct_dc_mlx5_ep_atomic_add32>, 
  ep_atomic_fadd32 = 0x51d2da0 <uct_dc_mlx5_ep_atomic_fadd32>, ep_atomic_swap32 = 0x51cd520 <uct_dc_mlx5_ep_atomic_swap32>, ep_atomic_cswap32 = 0x51cc040 <uct_dc_mlx5_ep_atomic_cswap32>, 
  ep_pending_add = 0x51bd9b0 <uct_dc_ep_pending_add>, ep_pending_purge = 0x51bded0 <uct_dc_ep_pending_purge>, ep_flush = 0x51c7840 <uct_dc_mlx5_ep_flush>, ep_fence = 0x5199290 <uct_base_ep_fence>, ep_check = 0x0, 
  ep_tag_eager_short = 0x0, ep_tag_eager_bcopy = 0x0, ep_tag_eager_zcopy = 0x0, ep_tag_rndv_zcopy = 0x0, ep_tag_rndv_cancel = 0x0, ep_tag_rndv_request = 0x0
@alinask
Copy link
Contributor

alinask commented May 25, 2017

The same reproduced with rc_mlx5 as well:

17:53:48 [ RUN      ] rc_mlx5/test_uct_stats.atomic_cswap64/0
17:53:48 [hpc-test-node2:6294 :2] Caught signal 11 (Segmentation fault: address not mapped to object at address (nil))
(gdb) bt
#0  0x00007ffff6b59a2d in pause () from /usr/lib64/libpthread.so.0
#1  0x00007ffff7adaa95 in ucs_debug_freeze () at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../src/ucs/debug/debug.c:735
#2  0x00007ffff7adc6c5 in ucs_error_freeze (message=0x7ffff7ff5d40 "Caught signal 11 (Segmentation fault: address not mapped to object at address (nil))",
    error_type=0x7ffff7b84409 "address not mapped to object") at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../src/ucs/debug/debug.c:854
#3  ucs_handle_error (error_type=error_type@entry=0x7ffff7b84409 "address not mapped to object", message=message@entry=0x7ffff7b841e3 "Caught signal %d (%s: %s%s)")
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../src/ucs/debug/debug.c:1018
#4  0x00007ffff7adca87 in ucs_debug_handle_error_signal (signo=signo@entry=11, cause=0x7ffff7b84409 "address not mapped to object", fmt=fmt@entry=0x7ffff7b844ab " at address %p")
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../src/ucs/debug/debug.c:958
#5  0x00007ffff7add20b in ucs_error_signal_handler (signo=11, info=0x7ffff7ff65f0, context=<optimized out>)
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../src/ucs/debug/debug.c:981
#6  <signal handler called>
#7  0x0000000000000000 in ?? ()
#8  0x000000000342e0d0 in ?? ()
#9  0x0000000000521760 in test_uct_stats::cleanup (this=0x342e0d0) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/uct/test_stats.cc:42
#10 0x000000000046875a in ucs::test_base::TearDownProxy (this=0x342e0e8) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/test.cc:185
#11 0x0000000000460193 in HandleSehExceptionsInMethodIfSupported<testing::Test, void> (location=0x72aa7a "TearDown()", method=<optimized out>, object=<optimized out>)
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest-all.cc:3562
#12 testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=object@entry=0x342e0d0, method=<optimized out>, location=location@entry=0x72aa7a "TearDown()")
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest-all.cc:3598
#13 0x000000000045418c in testing::Test::Run (this=this@entry=0x342e0d0) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest-all.cc:3643
#14 0x000000000045428c in testing::TestInfo::Run (this=0x2ba5140) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest-all.cc:3812
#15 0x00000000004543ef in testing::TestCase::Run (this=0x2b6b3f0) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest-all.cc:3930
#16 0x0000000000458f07 in testing::internal::UnitTestImpl::RunAllTests (this=this@entry=0x2a69730)
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest-all.cc:5802
#17 0x000000000045920b in RunAllTests (this=0x2a69730) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest-all.cc:5719
#18 HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (location=<optimized out>, method=<optimized out>, object=0x2a69730)
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest-all.cc:3562
#19 HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (location=0x72bf50 "auxiliary test code (environments or event listeners)",
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x459150 <testing::internal::UnitTestImpl::RunAllTests()>, object=0x2a69730)
    at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest-all.cc:3598
#20 testing::UnitTest::Run (this=<optimized out>) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest-all.cc:5416
#21 0x000000000040e9c2 in RUN_ALL_TESTS () at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/gtest.h:20059
#22 main (argc=1, argv=<optimized out>) at /scrap/jenkins/scrap/workspace/hpc-ucx-pr/label/hpc-test-node2/worker/3/contrib/../test/gtest/common/main.cc:79

http://hpc-master.lab.mtl.com:8080/job/hpc-ucx-pr/label=hpc-test-node2,worker=3/3687/console

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants