ThreadSanitizer broke between nightly build 07-06 and 07-07

[spacejam]

While working on my lock-free persistent b+ tree I updated my rustc and all of the sudden started having problems when running tsan.

λ uname -a

Linux eligos 4.12.6-1-ARCH #1 SMP PREEMPT Sat Aug 12 09:16:22 CEST 2017 x86_64 GNU/Linux 

λ cat src/main.rs

use std::thread;                                                                                                                                                                                                                                              

static mut ANSWER: i32 = 0;

fn main() {
    let t1 = thread::spawn(|| unsafe { ANSWER = 42 });
    unsafe {
        ANSWER = 24;
    }
    t1.join().ok();
}

λ RUSTFLAGS="-Z sanitizer=thread" rustup run nightly-2017-07-06-x86_64-unknown-linux-gnu cargo run --target x86_64-unknown-linux-gnu

    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs                                                               
     Running `target/x86_64-unknown-linux-gnu/debug/m`         
==================             
WARNING: ThreadSanitizer: data race (pid=23677)                
  Write of size 4 at 0x0017d33d9b84 by main thread:            
    #0 m::main /tmp/race/m/src/main.rs:8 (m+0xff7c)            
    #1 panic_unwind::__rust_maybe_catch_panic /checkout/src/libpanic_unwind/lib.rs:98 (m+0x2254c)                              
    #2 __libc_start_main ??:? (libc.so.6+0x204c9)              

  Previous write of size 4 at 0x0017d33d9b84 by thread T1:     
    #0 m::main::{{closure}} /tmp/race/m/src/main.rs:6 (m+0x1003c)                                                              
    #1 std::sys_common::backtrace::__rust_begin_short_backtrace<closure,()> /checkout/src/libstd/sys_common/backtrace.rs:136 (m+0xa775)
    #2 std::thread::{{impl}}::spawn::{{closure}}::{{closure}}<closure,()> /checkout/src/libstd/thread/mod.rs:364 (m+0xb54d)    
    #3 std::panic::{{impl}}::call_once<(),closure> /checkout/src/libstd/panic.rs:296 (m+0x9175)                                
    #4 std::panicking::try::do_call<std::panic::AssertUnwindSafe<closure>,()> /checkout/src/libstd/panicking.rs:479 (m+0xb719) 
    #5 panic_unwind::__rust_maybe_catch_panic /checkout/src/libpanic_unwind/lib.rs:98 (m+0x2254c)                              
    #6 std::panic::catch_unwind<std::panic::AssertUnwindSafe<closure>,()> /checkout/src/libstd/panic.rs:361 (m+0xa837)         
    #7 std::thread::{{impl}}::spawn::{{closure}}<closure,()> /checkout/src/libstd/thread/mod.rs:363 (m+0xb357)                 
    #8 alloc::boxed::{{impl}}::call_box<(),closure> /checkout/src/liballoc/boxed.rs:652 (m+0xeadf)                             
    #9 alloc::boxed::{{impl}}::call_once<(),()> /checkout/src/liballoc/boxed.rs:662 (m+0x1a76b)                                
    #10 std::sys_common::thread::start_thread /checkout/src/libstd/sys_common/thread.rs:21 (m+0x1a76b)                         
    #11 std::sys::imp::thread::{{impl}}::new::thread_start /checkout/src/libstd/sys/unix/thread.rs:84 (m+0x1a76b)              

  Location is global '<null>' at 0x000000000000 (m+0x0000002f5b84)                                                             

  Thread T1 (tid=23697, finished) created by main thread at:   
    #0 pthread_create /checkout/src/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:897 (m+0x56ff3)                              
    #1 std::sys::imp::thread::{{impl}}::new /checkout/src/libstd/sys/unix/thread.rs:72 (m+0x1a46e)                             
    #2 std::thread::spawn<closure,()> /checkout/src/libstd/thread/mod.rs:492 (m+0xa8dd)                                        
    #3 m::main /tmp/race/m/src/main.rs:6 (m+0xff6c)            
    #4 panic_unwind::__rust_maybe_catch_panic /checkout/src/libpanic_unwind/lib.rs:98 (m+0x2254c)                              
    #5 __libc_start_main ??:? (libc.so.6+0x204c9)              

SUMMARY: ThreadSanitizer: data race /tmp/race/m/src/main.rs:8 in m::main                                                       
==================             
ThreadSanitizer: reported 1 warnings                                                                                                                                                                                                                                         

λ RUSTFLAGS="-Z sanitizer=thread" rustup run nightly-2017-07-07-x86_64-unknown-linux-gnu cargo run --target x86_64-unknown-linux-gnu

    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs                                                                                                                                                                                              
     Running `target/x86_64-unknown-linux-gnu/debug/m`                                                                                                                                                                                                        
==26245==FATAL: ThreadSanitizer: failed to intercept pthread_mutex_lock

[kennytm]

Stack Overflow suggests that you need to increase your ulimit.

---

• version of 2017-07-06 = 1.20.0-nightly (3610a70ce 2017-07-05)
• version of 2017-07-07 = 1.20.0-nightly (696412de7 2017-07-06)

From 3610a70...696412d, I guess either #42899 (Switch to rust-lang-nursery/compiler-builtins) or #42816 (rustc: Implement stack probes for x86) is responsible.

---

No repro when running on macOS with a more recent nightly.

$ rustc -Z sanitizer=thread 1.rs

$ ./1
==================
WARNING: ThreadSanitizer: data race (pid=35728)
  Write of size 4 at 0x00010a2cf6c0 by thread T1:
    #0 _1::main::_$u7b$$u7b$closure$u7d$$u7d$::h2b2c1fdcf5ad676c <null>:204478040 (1:x86_64+0x100007758)
    #1 std::sys_common::backtrace::__rust_begin_short_backtrace::hf23e8bb2d4d2addb <null>:204478040 (1:x86_64+0x100001db5)
    #2 std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h1502a604e2389b0c <null>:204478040 (1:x86_64+0x100002ae5)
    #3 _$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::hb06834c90f50dd84 <null>:204478040 (1:x86_64+0x100000c45)
    #4 std::panicking::try::do_call::h04f3fa6c6abda8d6 <null>:204478040 (1:x86_64+0x100002c99)
    #5 __rust_maybe_catch_panic <null>:204478040 (1:x86_64+0x1000315bc)
    #6 std::panic::catch_unwind::hf462c68ec7a63481 <null>:204478040 (1:x86_64+0x100001e67)
    #7 std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::hcddefde833003ffd <null>:204478040 (1:x86_64+0x1000028d9)
    #8 _$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::h06c03c48d4910c94 <null>:204478040 (1:x86_64+0x100005cf4)
    #9 std::sys::imp::thread::Thread::new::thread_start::h431e210be94914a5 <null>:204478040 (1:x86_64+0x10000cf8b)

  Previous write of size 4 at 0x00010a2cf6c0 by main thread:
    #0 _1::main::hce80b422be4d604c <null>:204478040 (1:x86_64+0x10000769c)
    #1 __rust_maybe_catch_panic <null>:204478040 (1:x86_64+0x1000315bc)
    #2 start <null>:204479048 (libdyld.dylib:x86_64+0x5234)

  Location is global '_1::ANSWER::ha9074b9fd781a79e' at 0x00010a2cf6c0 (1+0x0001000476c0)

  Thread T1 (tid=8366680, running) created by main thread at:
    #0 pthread_create <null>:205529688 (libclang_rt.tsan_osx_dynamic.dylib:x86_64+0x7121)
    #1 std::sys::imp::thread::Thread::new::hc100d0d9e6164624 <null>:204478040 (1:x86_64+0x10000cc35)
    #2 std::thread::spawn::ha62bfd87b51b951c <null>:204478040 (1:x86_64+0x100001ee5)
    #3 _1::main::hce80b422be4d604c <null>:204478040 (1:x86_64+0x10000768c)
    #4 __rust_maybe_catch_panic <null>:204478040 (1:x86_64+0x1000315bc)
    #5 start <null>:204479048 (libdyld.dylib:x86_64+0x5234)

SUMMARY: ThreadSanitizer: data race (1:x86_64+0x100007758) in _1::main::_$u7b$$u7b$closure$u7d$$u7d$::h2b2c1fdcf5ad676c
==================
ThreadSanitizer: reported 1 warnings
Abort trap: 6

$ rustc -vV
rustc 1.21.0-nightly (7ac979d8c 2017-08-16)
binary: rustc
commit-hash: 7ac979d8cbe97241fd39f4037e1d4069caaff4d2
commit-date: 2017-08-16
host: x86_64-apple-darwin
release: 1.21.0-nightly
LLVM version: 4.0

[spacejam]

λ ulimit -v
unlimited

[est31]

@spacejam can you reproduce on latest nightlies? The nightly you cite had bugs with certain kernel versions, but they have been fixed since: #43102

[spacejam]

@est31 still an issue for me on the 8/20 nightly
λ RUSTFLAGS="-Z sanitizer=thread" rustup run nightly-2017-08-20-x86_64-unknown-linux-gnu cargo run --target x86_64-unknown-linux-gnu

Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs                               
Running `target/x86_64-unknown-linux-gnu/debug/tsan`                                      
==18176==FATAL: ThreadSanitizer: failed to intercept pthread_mutex_lock 

[kennytm]

Repro on Debian. Looks like it only affects Linux, but not macOS (which uses a dylib TSan instead of staticlib).

Given these I strongly suspect #42816 is the cause (cc @alexcrichton):

rust/src/librustc_trans/attributes.rs

Lines 80 to 86 in 757b7ac

	// Currently stack probes seem somewhat incompatible with the address
	// sanitizer. With asan we're already protected from stack overflow anyway
	// so we don't really need stack probes regardless.
	match ccx.sess().opts.debugging_opts.sanitizer {
	Some(Sanitizer::Address) => return,
	_ => {}
	}

Is it possible to turn off stack-probes with a -Z flag to check? I don't think sanitizers can work with JSON targets.

[spacejam]

This was fixed on nightly-2017-08-28-x86_64-unknown-linux-gnu (I binary searched, 2017-08-27 was broken still).

Thanks!