defmt crate doesn't compile on macOS

[BriocheBerlin]

Updated description:
defmt 0.1.3 compiles on macOS but the git version (0.2.0) doesn't
repro:

$ cargo new --bin hello
$ cd hello
$ cargo add defmt --vers 0.1.3
$ cargo run
$ # ^ doesn't work if you use the git version

---

When trying to get started to write compile fail tests , running
$ cargo test --test ui
didn't work for me using a mac. I'd always get this unhelpful error:

LLVM ERROR: Global variable '{"package":"defmt","tag":"defmt_prim","data":"{=i8}","disambiguator":"2773326613612971761"}' has an invalid section specifier '.defmt.prim.{"package":"defmt","tag":"defmt_prim","data":"{=i8}","disambiguator":"2773326613612971761"}': mach-o section specifier requires a segment whose length is between 1 and 16 characters.
error: could not compile `defmt` 

Passing more flags like this did help:
$ cargo test -p defmt --all-features --test ui

This makes sense when looking at ci.yml

[japaric]

@BriocheBerlin could you test this on macOS? I'd like to figure out if defmt fails to compile natively or if this problem is due to the single workspace configuration used in this repository.

$ cargo new --bin hello
$ cd hello
$ # or manually add 'defmt = "0.1.3"' to Cargo.toml
$ cargo add defmt 
$ # does this succeed?
$ cargo run

[BriocheBerlin]

$ cargo run succeeded.

[japaric]

@BriocheBerlin could you run cargo test --test ui -j1 in this repo and report which crate fails to build? That command builds one crate at a time rather than in parallel.

[BriocheBerlin]

So it seems that in the working version on the hello project the command for building the defmt crate is very similar. However it uses a different src/lib.rs file which is local <more_path>/.cargo/registry/src/github.mirror.nvdadr.com-1ecc6299db9ec823/defmt-0.1.3/src/lib.rs. It is different to the recent file src/lib.rs however simply switching it out does not work. Also it seems old(?) because doesn't have the InternalFormatter.

[japaric]

hmm, maybe the macOS issue was introduced recently so it's in the git version but not on crates.io?
could you try these two:

$ cargo new --bin hello
$ cd hello

$ # or add 'defmt = { git = "https://github.com/knurling-rs/defmt" }' to Cargo.toml dependencies
$ cargo add defmt --git https://github.com/knurling-rs/defmt

$ # does this succeed?
$ cargo run

$ git clone https://github.com/knurling-rs/defmt
$ cd defmt
$ git checkout defmt-v0.1.3

$ # do these produce the llvm error?
$ cargo build
$ cargo test

[BriocheBerlin]

defmt-v0.1.3 works without problems. However the hello thing produces this interesting error:

Caused by:
  failed to load source for dependency `defmt`
Caused by:
  Unable to update https://github.com/knurling-rs/defmt
Caused by:
  failed to find branch `master`
Caused by:
  cannot locate remote-tracking branch 'origin/master'; class=Reference (4); code=NotFound (-3)

But of course there is no master anymore.

[japaric]

oh, yes. my bad it should be

defmt = { git = "https://github.com/knurling-rs/defmt", branch = "main" }

in the Cargo.toml

[BriocheBerlin]

Ah, doesn't work. So (besides different src/lib.rs files) the resulting commands are slightly different.
Working version (using defmt = "0.1.3" in Cargo.toml):

rustc --crate-name defmt 
--edition=2018 
<more_path>/.cargo/registry/src/github.mirror.nvdadr.com-1ecc6299db9ec823/defmt-0.1.3/src/lib.rs 
--error-format=json 
--json=diagnostic-rendered-ansi 
--crate-type lib 
--emit=dep-info,metadata,link 
-C embed-bitcode=no 
-C debuginfo=2 
-C metadata=4f3b6bbbb29948b6 
-C extra-filename=-4f3b6bbbb29948b6 
--out-dir <more_path>/hello/target/debug/deps 
-L dependency=<more_path>/hello/target/debug/deps 
--extern defmt_macros=<more_path>/hello/target/debug/deps/libdefmt_macros-f658b9f3b197fd12.dylib 
--extern heapless=<more_path>/hello/target/debug/deps/libheapless-9846ae8300a43c3e.rmeta 
--cap-lints allow 
-L <more_path>/hello/target/debug/build/defmt-d1e4ca37c018f197/out

With defmt = { git = "https://github.com/knurling-rs/defmt", branch = "main" } it doesn't have
--extern heapless=<path>
When running on defmt as is, this line is different:
-L dependency=<more_path>/hello/target/debug/deps
Instead I got:
-C incremental=<more_path>/defmt/target/debug/incremental

[japaric]

Then this more serious: as things are defmt 0.2 won't compile on macOS which is going to hinder adoption: using defmt in a no-std, non-embedded crate would make the crate unusable on macOS.

I'll put this on the list of blockers for 0.2.0. cc @jonas-schievink

[jonas-schievink]

The invalid section names were already used in defmt 0.1 though? Or is there another error?

[japaric]

I think this may have been caused by PR #291 . After the changes in the macros we emit link_section on x86_64 macOS -- before we emitted fetch_add_string_index on x86_64 macOS.

[BriocheBerlin]

That would explain why in the working src/lib.rs file it was always target_arch = "x86_64" instead of feature = "unstable-test".

[BriocheBerlin]

The last commit where $ cargo build works for me is 04cf573.
(But $ cargo test fails, again complaining about missing InternalFormatter).

[japaric]

we looked into this. the llvm error can be fixed by changing the linker section names we use.
currently we use .defmt.print.<LONG JSON STRING HERE>
on macOS the longest linker section name can be .defmt,0123456789ABCDEF (note comma after .defmt not period)
we should adjust the macros to generate a linker section name like that
instead of we should use digest / hash of that and truncate it to not exceed the 16 character limits

[japaric]

for starters this proc-macro needs to be changed (but probably others also need to be changed to work on macOS):

defmt/macros/src/lib.rs

Line 987 in eb8c3b1

let section = format!(".defmt.prim.{}", sym);

[japaric]

we should also build and run the following std program on linux, macos and windows to shake out any other linker section name related bugs:

// Cargo.toml uses defmt via a path dependency

fn main() {
    defmt::info!("hello");
}

#[defmt::global_logger]
struct Logger;

unsafe impl defmt::Logger for Logger {
    fn acquire() -> Option<core::ptr::NonNull<dyn defmt::Write>> {
        None
    }
    unsafe fn release(writer: core::ptr::NonNull<dyn defmt::Write>) {}
}

defmt::timestamp!("{=u32}", 0);

[Urhengulas]

This issue seems resolved, right?

[jonas-schievink]

Yup!