VENDORIZE: add feature vendored #498
Conversation
Add the feature vendored to turn on that feature in libbpf-sys see: libbpf/libbpf-sys#59 This feature will get libbpf-sys to do all the static compilation rather than the application developer statically compiling these libraries themselves, or relying on their distribution to provide static versions of libelf and zlib. The "static" feature still exists for those who still wish it, as compiling libelf adds significant compilation time. Vendored was added to libbpf-sys due to some major distributions no longer shipping statically compiled versions of libelf (eg Fedora). CHANGELOG NOTE: feature "vendored" allows libelf and zlib to be statically linked. Completely static binaries can now be compiled via the command `RUSTFLAGS='-C target-feature=+crt-static' cargo build --target x86_64-unknown-linux-gnu`. This feature will not work with musl. signed-off-by: Michael Mullin <mmullin@halcyon.ai>
|
Thanks for the pull request! This is fine in isolation, but I feel like the perceived
What do you think? |
Something like? Where: It's a bit annoying to need to set The weird case is actually static, as libbpf-sys would still be providing the libbpf.a static lib. There is currently no way to get a static libbpf.a library external to libbpf-sys. I don't particularly like this due to the default-features=false, I'd rather stick to default, novendor, static, and vendored. vendored could be renamed to vendored-libbpf-deps to remove confusion about vendored vs novendor. I understand why you would want "all external dynamic libs" to be the default though, as it's more logical to start from zero and add, rather than start from something and remove. it's sort of like counting 0,1,2,3 rather than -1,0,1,2. We could ask @alexforster if he would be amenable to changing libbpf-sys' flags around such that afxdp, rebpf, and xsk-rs all seem to be dead projects, so I don't think there would be too much problem with this I am prepared to do this work, Alex would need to give approval because of the other (possibly dead) projects libbpf-rs could then mimic the libbpf-sys features. |
Yes, something like that.
Indeed, that is unfortunate and not nice. I was trying to keep changes to
I was honestly just trying to preserve the existing behavior, at least how I understand it.
In my opinion vendoring and linking are completely orthogonal. Conceptually we can vendor a library and then link it statically or dynamically or we can link it statically or dynamically from the system. In practice dynamic linking when vendoring may not be that practical due to tooling issues (search path and whatever other complications there may be), though I don't believe there is anything inherently undesirable or impossible about it. With that in mind, in my opinion, and given the logic we already have, I'd envision: [features]
default = ['vendored-libbpf'] # current behavior, we can keep it or change it; just a proposal
vendored = ['vendored-libbpf', 'vendored-elf', 'vendored-zlib']
vendored-libbpf = []
vendored-elf = []
vendored-zlib = []
static = ['static-libbpf', 'static-elf', 'static-zlib']
static-libbpf = []
static-elf = []
static-zlib = []This way, in my opinion there is very little confusion and everything is selectable without restriction. There would also be no negation. Why would anybody link
I don't believe this would necessarily be breaking. We could conceivably keep the existing features around and just deprecate them (that may be most appropriate for a One more thing (very general comment): I don't know if we necessarily need to plumb all those features through Also, yet more generally: I found https://kornel.ski/rust-sys-crate to be a good summary of good practices in this realm. Anyway, let's wait for @alexforster's take. Edit:
👍 |
Does "vendoring" mean providing as part of Rust library? I'm just wondering what's the point of vendoring, if one doesn't link against it? I might be missing something. But for the above, I think it's an overkill in terms of all the options available to the users. While theoretically there might be some combination of statically linking one subset of libraries while dynamically linking another, I think in practice it's more all-or-nothing: one either goes for dynamic linking and then libelf, libz, and libbpf should be .so and linked dynamically, or you go all-in on static linking and would like libelf.a+libz.a+libbpf.a all statically linked to not have to rely on global environment providing them (and at good versions). Also, if I was a user of libbpf/libbpf-rs. I wouldn't want to care all that much that there is elf and zlib (and maybe some other one in the future) dependencies to libbpf. I'd like to say "just please statically link all the needed stuff so I don't have to care or think about this". So I wonder if maybe we should just provide 2 or 3 options: no-vendoring (all dynamically linked), all vendoring (libbpf+libz+libelf all vendored and statically linked), and maybe for backwards compat/compile time saving, libbpf-only-vendoring (current default, as far as I understand). Naming are completely for demo purposes, of course. In short, too many options and knobs might actually hurt :) Especially for Rust folks that are not that familiar with horrors of C/C++ world of linking hell. |
|
I am in favour of a simplistic "we provide no libraries, get shared libs from your distribution" vs "we provide everything statically linked" The super easy solution is: The proper solution is: |
I'd be okay with that. |
Sorry, I meant to quote both the "super easy solution" and the "proper solution". I am fine with both, as long as we don't have |
|
Haven't read this whole thread but in general I'm happy to go along with whatever y'all decide. One caveat: we do need to keep the existing functionality of the |
|
@mmullin-halcyon are you still waiting on input on this PR? |
No. I simply had a busy weekend and haven't had time to dedicate. I will be on this tonight. Sorry. |
Add the feature vendored to turn on that feature in libbpf-sys see: libbpf/libbpf-sys#59
This feature will get libbpf-sys to do all the static compilation rather than the application developer statically compiling these libraries themselves, or relying on their distribution to provide static versions of libelf and zlib. The "static" feature still exists for those who still wish it, as compiling libelf adds significant compilation time.
Vendored was added to libbpf-sys due to some major distributions no longer shipping statically compiled versions of libelf (eg Fedora).
CHANGELOG NOTE: feature "vendored" allows libelf and zlib to be statically linked. Completely static binaries can now be compiled via the command
RUSTFLAGS='-C target-feature=+crt-static' cargo build --target x86_64-unknown-linux-gnu.CHANGELOG NOTE: Feature "vendored" will not work with musl.
Signed-off-by: Michael Mullin mmullin@halcyon.ai