-
-
Notifications
You must be signed in to change notification settings - Fork 13.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
setup-hooks/reproducible-builds.sh: NIX_OUTPATH_USED_AS_RANDOM_SEED #216016
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
For reproducibility, nixpkgs sets `-frandom-seed` to the first 10 characters of the outpath of the derivation being built. This PR allows to manually select from which outpath the 10-character prefix is taken, by setting `NIX_OUTPATH_USED_AS_RANDOM_SEED` as a derivation attribute (or directly as an environment variable). For an example use of this functionality, see: #209870
ghost
mentioned this pull request
Feb 12, 2023
4 tasks
trofi
approved these changes
Feb 18, 2023
This was accidentally merged into master? |
Gah, you are right! Thank you for a prompt revert! |
Re-targeted against |
ghost
deleted the
pr/setup/NIX_OUTPATH_USED_AS_RANDOM_SEED
branch
February 18, 2023 16:46
tm-drtina
pushed a commit
to awakesecurity/nixpkgs
that referenced
this pull request
Apr 27, 2024
#### Summary By default, when you type `make`, GCC will compile itself three times. This PR inhibits that behavior by configuring GCC with `--disable-bootstrap`, and reimplements the triple-rebuild using Nix rather than `make`/`sh`. #### Immediate Benefits - Allow `gcc11` and `gcc12` on `aarch64` (without needing new `bootstrapFiles`) - Faster stdenv rebuilds: the third compilation of gcc (i.e. stageCompare) is no longer a `drvInput` of the final stdenv. This allows Nix to build stageCompare in parallel with the rest of nixpkgs instead of in series. - No more copying `libgcc_s` out of the bootstrap-files or other derivations - No more Frankenstein compiler: the final gcc and the libraries it links against (mpfr, mpc, isl, glibc) are all built by the same compiler (xgcc) instead of a mixture of the bootstrapFiles' compiler and xgcc. - No more [static lib{mpfr,mpc,gmp,isl}.a hack] - Many other small `stdenv` hacks eliminated - `gcc` and `clang` share the same codepath for more of `cc-wrapper`. #### Future Benefits - This should allow using a [foreign] `bootstrap-files` so long as `hostPlatform.canExecute bootstrapFiles`. - This should allow each of the libraries that ship with `gcc` (lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty, offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be built in separate (one-liner) derivations which `inherit src;` from `gcc`, much like NixOS#132343 #### Incorporates - NixOS#210004 - NixOS#36948 (unreverted) - NixOS#210325 - NixOS#210118 - NixOS#210132 - NixOS#210109 - NixOS#213909 - NixOS#216136 - NixOS#216237 - NixOS#210019 - NixOS#216232 - NixOS#216016 - NixOS#217977 - NixOS#217995 #### Closes - Closes NixOS#108305 - Closes NixOS#108111 - Closes NixOS#201254 - Closes NixOS#208412 #### Credits This project was made possible by three important insights, none of which were mine: 1. @Ericson2314 was the first to advocate for this change, and probably the first to appreciate its advantages. Nix-driven (external) bootstrap is "cross by default". 2. @trofi has figured out a lot about how to get gcc to not mix up the copy of `libstdc++` that it depends on with the copy that it builds, by moving the `bootstrapFiles`' `libstdc++` into a [versioned directory]. This allows a Nix-driven bootstrap of gcc without the final gcc would still having references to the `bootstrapFiles`. 3. Using the undocumented variable [`user-defined-trusted-dirs`] when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it uses a completely different and totally special set of rules for finding `libgcc_s.so`. This trick is the only way we can put `libgcc_s.so` in its own separate outpath without creating circular dependencies or dependencies on the bootstrapFiles. I would never have guessed to use this (or that it existed!) if it were not for a [comment in guix] which @Mic92 [mentioned]. My own role in this PR was basically: being available to go on a coding binge at an opportune moment, so we wouldn't waste a [crisis]. [aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938 [amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857 [nonexistent sysroot]: NixOS#210004 [versioned directory]: NixOS#209054 [`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html [comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253 [mentioned]: NixOS#210112 (comment) [crisis]: NixOS#108305 [foreign]: NixOS#170857 (comment) [static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes
For reproducibility, nixpkgs sets
-frandom-seed
to the first 10 characters of the outpath of the derivation being built.This PR allows to manually select from which outpath the 10-character prefix is taken, by setting
NIX_OUTPATH_USED_AS_RANDOM_SEED
as a derivation attribute (or directly as an environment variable).For an example use of this functionality, see #209870
Things done
sandbox = true
set innix.conf
? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"
. Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/
)