ociTools.buildContainer erroring with "path [foo] is not in the Nix store" #316652

tomfitzhenry · 2024-06-02T12:20:43Z

Describe the bug

pkgs.ociTools.buildContainer is failing with errors such as error: path '/nix/store/306znyj77fv49kwnkpxmb0j2znqpa8bj-bash-5.2p26/bin/bash' is not in the Nix store.

This example in the documentation ( https://nixos.org/manual/nixpkgs/unstable/#ssec-pkgs-ociTools-buildContainer-examples ) fails in this way, as well as when I try to integrate this into my NixOS configuration.

Steps To Reproduce

Follow https://nixos.org/manual/nixpkgs/unstable/#ssec-pkgs-ociTools-buildContainer-examples , e.g.

$ cat default.nix
{ ociTools, lib, bash }:
ociTools.buildContainer {
  args = [
    (lib.getExe bash)
  ];

  readonly = false;
}

$ nix-build --expr '(import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/ad57eef4ef0659193044870c731987a6df5cf56b.tar.gz") {}).callPackage ./default.nix {}'
these 3 derivations will be built:
  /nix/store/18fgd6sz9q8d9r9xwbxkz4nid8kpq2ah-runtime-deps.drv
  /nix/store/5vjjr5sxibh9r7llxs2n03qzjn4r6p56-config.json.drv
  /nix/store/1v6mjrp8nj9a1xns1i75r3zdxzab2kqp-join.drv
building '/nix/store/5vjjr5sxibh9r7llxs2n03qzjn4r6p56-config.json.drv'...
error: path '/nix/store/306znyj77fv49kwnkpxmb0j2znqpa8bj-bash-5.2p26/bin/bash' is not in the Nix store

Expected behavior

I expected an OCI image to be built.

Notify maintainers

@ShamrockLee @aib

Add a 👍 reaction to issues you find important.

The text was updated successfully, but these errors were encountered:

ShamrockLee · 2024-06-02T14:24:53Z

Thanks for pinning, but I haven't worked on ociTools before and am unsure if I could help.

ociTools.buildContainer sounds fantastic, and I hope to eventually get it to work with Apptainer and SingularityCE (the container packages I'm maintaining).

Cc: @SomeoneSerge (who might also be interested)

ShamrockLee · 2024-06-02T14:35:46Z

I cannot reproduce this issue on my laptop.

Reproducer:

nix-repl> lib = import ./lib                                                                                  
nix-repl> pkgs = import ./. { config = { allowAliases = false; checkMeta = true; }; }                         

nix-repl> oci-example = pkgs.ociTools.buildContainer { args = [ (lib.getExe pkgs.bash) ]; readonly = false; } 

nix-repl> oci-example
«derivation /nix/store/hcw309zyix75sym0mhz8hn2kfi8kx4sw-join.drv»

$ nix-build --no-out-link /nix/store/hcw309zyix75sym0mhz8hn2kfi8kx4sw-join.drv

aib · 2024-06-02T19:40:47Z

I... am not sure what's going on.

I stopped my nix adventure when I was still deeply confused by numerous similar-but-different concepts despite completing multiple tutorials and spending several days practicing. As such, I don't know what nix-build is doing, or how it's related to default.nix or the repl output in the examples above.

I would love to help if I could do so in any way, though. (I'm even willing to go through another tutorial, if anyone has a good one handy.)

ShamrockLee · 2024-06-02T20:29:35Z

Update:

I tried using an inlined version of @tomfitzhenry's reproducer and saw the same error.

nix-build --expr '(import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/ad57eef4ef0659193044870c731987a6df5cf56b.tar.gz") { }).callPackage ({ ociTools, lib, bash }: ociTools.buildContainer { args = [ (lib.getExe bash) ]; readonly = false; }) { }'

Something is wrong, but I wonder why the two reproducers would behave differently.

aib · 2024-06-03T10:49:50Z

Looks like it's due to the writeReferencesToFile -> writeClosure change in e9fd438 . I'll continue applying my non-nix-specific skills to try to debug further...

aib · 2024-06-03T11:06:29Z

This is probably obvious for others, but:

writeReferencesToFile's command takes the file graph:

/nix/store/4vzal97iq3dmrgycj8r0gflrh51p8w1s-bash-5.2p26

2
/nix/store/4vzal97iq3dmrgycj8r0gflrh51p8w1s-bash-5.2p26
/nix/store/ksk3rnb0ljx8gngzk19jlmbjyvac4hw6-glibc-2.38-44
/nix/store/d8w5qfswmgxcjqwnmqw2v9r8amrvdlpb-xgcc-13.2.0-libgcc

0
/nix/store/krqp9wj3rgalmqv04y0sqw987mxsnddn-libidn2-2.3.7

2
/nix/store/krqp9wj3rgalmqv04y0sqw987mxsnddn-libidn2-2.3.7
/nix/store/vqvbn2z8wyrjwvayjb2vy5krhh1kis9b-libunistring-1.1
/nix/store/ksk3rnb0ljx8gngzk19jlmbjyvac4hw6-glibc-2.38-44

3
/nix/store/d8w5qfswmgxcjqwnmqw2v9r8amrvdlpb-xgcc-13.2.0-libgcc
/nix/store/krqp9wj3rgalmqv04y0sqw987mxsnddn-libidn2-2.3.7
/nix/store/ksk3rnb0ljx8gngzk19jlmbjyvac4hw6-glibc-2.38-44
/nix/store/vqvbn2z8wyrjwvayjb2vy5krhh1kis9b-libunistring-1.1

1
/nix/store/vqvbn2z8wyrjwvayjb2vy5krhh1kis9b-libunistring-1.1

And converts it to the file $out:

/nix/store/4vzal97iq3dmrgycj8r0gflrh51p8w1s-bash-5.2p26
/nix/store/d8w5qfswmgxcjqwnmqw2v9r8amrvdlpb-xgcc-13.2.0-libgcc
/nix/store/krqp9wj3rgalmqv04y0sqw987mxsnddn-libidn2-2.3.7
/nix/store/ksk3rnb0ljx8gngzk19jlmbjyvac4hw6-glibc-2.38-44
/nix/store/vqvbn2z8wyrjwvayjb2vy5krhh1kis9b-libunistring-1.1

writeClosure uses $NIX_ATTRS_JSON_FILE and jq to do the same. The problem is, runCommand doesn't like the "${pkgs.bash}/bin/bash" passed in exportReferencesGraph.graph.

(I guess I'm taking notes here in this comment)

ShamrockLee · 2024-06-03T11:43:04Z

Looks like it's due to the writeReferencesToFile -> writeClosure change in e9fd438 .

I'm sorry for any trouble the change might cause. I was the author of this change. The corresponding PR is #178717.

writeReferencesToFile takes a graph:

It took me quite some time to understand when working on the writeReferencesToFile -> writeClosure refactoring. To simplify things, writeClosure utilizes the new, JSON-based exportReferencesGraph interface when specifying __structuredAttrs = true. Here is the discussion thread: #178717 (comment)

aib · 2024-06-03T12:07:33Z

One thing I don't understand is why we get the "x is not in the Nix store" error when x is an actual file under /nix/store.

As for buildContainer, are we sure we want to be parsing args as a list of dependencies? The first element is a command, for sure, but how do we know the rest are filenames, let alone files that need to exist and be copied into the rootfs?

I think we need a separate "dependencies" parameter where pkgs.bash can go. The Docker builder seems to have... copyToRoot? Is that what a derivation means?

ShamrockLee · 2024-06-03T12:37:36Z

It turns out that the samples (tests.trivial-builders.references.samples) of the test case tests.trivial-builders.references doesn't cover lib.getExe-related use cases. The error appears as soon as such a sample is added.

ShamrockLee · 2024-06-03T12:53:20Z

One thing I don't understand is why we get the "x is not in the Nix store" error when x is an actual file under /nix/store.

I think it's a bug in the structured exportReferencesGraph interface of Nix. We can reproduce it simply by using the interface:

Legacy interface, `outPath`, successful build

nix-build --no-out-link -E "let lib = import ./lib; pkgs = import ./. { }; path = pkgs.hello.outPath; in pkgs.runCommandLocal ''reference-graph-legacy'' { exportReferencesGraph = [ ''graph'' path ]; } ''cat ./graph | tee \$out''"

this derivation will be built:
  /nix/store/diigmg6bqxpixcj3dzzqavmc6783pjs1-reference-graph-legacy.drv
building '/nix/store/diigmg6bqxpixcj3dzzqavmc6783pjs1-reference-graph-legacy.drv'...
/nix/store/1q9vc0lq7qjlfjz47mfmlzdf86c543jy-xgcc-13.2.0-libgcc

0
/nix/store/ic63ay0py10fyryaw7345k4ps32da33w-libidn2-2.3.7

2
/nix/store/ic63ay0py10fyryaw7345k4ps32da33w-libidn2-2.3.7
/nix/store/yfp7dr8m7zi7kxk49wd714gwvhb105hf-libunistring-1.1
/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52

3
/nix/store/1q9vc0lq7qjlfjz47mfmlzdf86c543jy-xgcc-13.2.0-libgcc
/nix/store/ic63ay0py10fyryaw7345k4ps32da33w-libidn2-2.3.7
/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52
/nix/store/yfp7dr8m7zi7kxk49wd714gwvhb105hf-libunistring-1.1

1
/nix/store/yfp7dr8m7zi7kxk49wd714gwvhb105hf-libunistring-1.1
/nix/store/yh6x9ia5kxxw3w90vkb09vqgfvhb416k-hello-2.12.1

2
/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52
/nix/store/yh6x9ia5kxxw3w90vkb09vqgfvhb416k-hello-2.12.1
/nix/store/ik4m89xg09jx21fkjh54varbj450m5dp-reference-graph-legacy

Legacy interface, `getExe`, successful build

ix-build --no-out-link -E "let lib = import ./lib; pkgs = import ./. { }; path = lib.getExe pkgs.hello; in pkgs.runCommandLocal ''reference-graph-legacy'' { exportReferencesGraph = [ ''graph'' path ]; } ''cat ./graph | tee \$out''"

this derivation will be built:
  /nix/store/rlzp3ilxh7xghsn22vnjblccijhg7c29-reference-graph-legacy.drv
building '/nix/store/rlzp3ilxh7xghsn22vnjblccijhg7c29-reference-graph-legacy.drv'...
/nix/store/1q9vc0lq7qjlfjz47mfmlzdf86c543jy-xgcc-13.2.0-libgcc

0
/nix/store/ic63ay0py10fyryaw7345k4ps32da33w-libidn2-2.3.7

2
/nix/store/ic63ay0py10fyryaw7345k4ps32da33w-libidn2-2.3.7
/nix/store/yfp7dr8m7zi7kxk49wd714gwvhb105hf-libunistring-1.1
/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52

3
/nix/store/1q9vc0lq7qjlfjz47mfmlzdf86c543jy-xgcc-13.2.0-libgcc
/nix/store/ic63ay0py10fyryaw7345k4ps32da33w-libidn2-2.3.7
/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52
/nix/store/yfp7dr8m7zi7kxk49wd714gwvhb105hf-libunistring-1.1

1
/nix/store/yfp7dr8m7zi7kxk49wd714gwvhb105hf-libunistring-1.1
/nix/store/yh6x9ia5kxxw3w90vkb09vqgfvhb416k-hello-2.12.1

2
/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52
/nix/store/yh6x9ia5kxxw3w90vkb09vqgfvhb416k-hello-2.12.1
/nix/store/h9bigga3j2ah1dpakjz1bpsyzy6iqvf2-reference-graph-legacy

Structured interface, `outPath`, successful build

nix-build --no-out-link -E "let lib = import ./lib; pkgs = import ./. { }; path = pkgs.hello.outPath; in pkgs.runCommandLocal ''reference-graph-structured'' { __structuredAttrs = true; nativeBuildInputs = [ pkgs.buildPackages.jq ]; exportReferencesGraph.graph = [ path ]; } ''jq . \$NIX_ATTRS_JSON_FILE | tee \$out''"

this derivation will be built:
  /nix/store/6wrl680daaffrm8hy4ahlkslv7dlq28q-reference-graph-structured.drv
building '/nix/store/6wrl680daaffrm8hy4ahlkslv7dlq28q-reference-graph-structured.drv'...
structuredAttrs is enabled
{
  "allowSubstitutes": false,
  "buildCommand": "jq . $NIX_ATTRS_JSON_FILE | tee $out",
  "buildInputs": [],
  "builder": "/nix/store/306znyj77fv49kwnkpxmb0j2znqpa8bj-bash-5.2p26/bin/bash",
  "cmakeFlags": [],
  "configureFlags": [],
  "depsBuildBuild": [],
  "depsBuildBuildPropagated": [],
  "depsBuildTarget": [],
  "depsBuildTargetPropagated": [],
  "depsHostHost": [],
  "depsHostHostPropagated": [],
  "depsTargetTarget": [],
  "depsTargetTargetPropagated": [],
  "doCheck": false,
  "doInstallCheck": false,
  "enableParallelBuilding": true,
  "enableParallelChecking": true,
  "enableParallelInstalling": true,
  "env": {},
  "exportReferencesGraph": {
    "graph": [
      "/nix/store/yh6x9ia5kxxw3w90vkb09vqgfvhb416k-hello-2.12.1"
    ]
  },
  "graph": [
    {
      "closureSize": 159560,
      "narHash": "sha256:10q8iyvfmpfck3yiisnj1j8vp6lq3km17r26sr95zpdf9mgmk69s",
      "narSize": 159560,
      "path": "/nix/store/1q9vc0lq7qjlfjz47mfmlzdf86c543jy-xgcc-13.2.0-libgcc",
      "references": [],
      "valid": true
    },
    {
      "closureSize": 2190120,
      "narHash": "sha256:0mfvcix1h7prq9wwpwfbwyj56pkdm3w05i69zwvd911fg27w7ccy",
      "narSize": 361136,
      "path": "/nix/store/ic63ay0py10fyryaw7345k4ps32da33w-libidn2-2.3.7",
      "references": [
        "/nix/store/ic63ay0py10fyryaw7345k4ps32da33w-libidn2-2.3.7",
        "/nix/store/yfp7dr8m7zi7kxk49wd714gwvhb105hf-libunistring-1.1"
      ],
      "valid": true
    },
    {
      "closureSize": 32473824,
      "narHash": "sha256:1ljsk162rvyhjdsfbiiq26lsm8ahz68y1g366r6wwbq31d1aanah",
      "narSize": 30124144,
      "path": "/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52",
      "references": [
        "/nix/store/1q9vc0lq7qjlfjz47mfmlzdf86c543jy-xgcc-13.2.0-libgcc",
        "/nix/store/ic63ay0py10fyryaw7345k4ps32da33w-libidn2-2.3.7",
        "/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52"
      ],
      "valid": true
    },
    {
      "closureSize": 1828984,
      "narHash": "sha256:1akas0jc72xbc8fz1a8ja1gk5ghh1jqm9s1p46p67sg2bcjfiwzg",
      "narSize": 1828984,
      "path": "/nix/store/yfp7dr8m7zi7kxk49wd714gwvhb105hf-libunistring-1.1",
      "references": [
        "/nix/store/yfp7dr8m7zi7kxk49wd714gwvhb105hf-libunistring-1.1"
      ],
      "valid": true
    },
    {
      "closureSize": 32700384,
      "narHash": "sha256:1s8ax2azcifcnxsn0ny1gk75hd6hfmm3l5nd56xpsxmya2gwnyg9",
      "narSize": 226560,
      "path": "/nix/store/yh6x9ia5kxxw3w90vkb09vqgfvhb416k-hello-2.12.1",
      "references": [
        "/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52",
        "/nix/store/yh6x9ia5kxxw3w90vkb09vqgfvhb416k-hello-2.12.1"
      ],
      "valid": true
    }
  ],
  "mesonFlags": [],
  "name": "reference-graph-structured",
  "nativeBuildInputs": [
    "/nix/store/n5chc8i5za5jrr4biwwgx7a96p0a7aq9-jq-1.7.1-dev"
  ],
  "outputs": {
    "out": "/nix/store/l30rfsn0lcbnxrd248s40ak22rvrqkm3-reference-graph-structured"
  },
  "passAsFile": [
    "buildCommand"
  ],
  "patches": [],
  "preferLocalBuild": true,
  "propagatedBuildInputs": [],
  "propagatedNativeBuildInputs": [],
  "stdenv": "/nix/store/rg6agzlb4gcl4w2126m16qm4mgr8jgi6-stdenv-linux",
  "strictDeps": false,
  "system": "x86_64-linux"
}
/nix/store/l30rfsn0lcbnxrd248s40ak22rvrqkm3-reference-graph-structured

Structured interface, `getExe`, failed build

nix-build --no-out-link -E "let lib = import ./lib; pkgs = import ./. { }; path = lib.getExe pkgs.hello; in pkgs.runCommandLocal ''reference-graph-structured'' { __structuredAttrs = true; nativeBuildInputs = [ pkgs.buildPackages.jq ]; exportReferencesGraph.graph = [ path ]; } ''jq . \$NIX_ATTRS_JSON_FILE | tee \$out''"

this derivation will be built:
  /nix/store/n4a18ikia9x5pa5x4hsjvsgb1gnv66wh-reference-graph-structured.drv
error: path '/nix/store/yh6x9ia5kxxw3w90vkb09vqgfvhb416k-hello-2.12.1/bin/hello' is not in the Nix store

ShamrockLee · 2024-06-03T13:00:12Z

The upstream seems to have noticed this. For details, see NixOS/nix#10549 and NixOS/nix#10565.

aib · 2024-06-03T17:34:20Z

Ahh great catch! Thanks.

I still think it will be a good idea to have a dedicated "dependencies" or "copyToRoot" field for this, given args is a command array.

tomfitzhenry added the 0.kind: bug label Jun 2, 2024

ShamrockLee mentioned this issue Jun 3, 2024

Test "fix exportReferencesGraph when given store subpath" NixOS/nix#10565

Open

ShamrockLee mentioned this issue Jun 4, 2024

Fix exportReferencesGraph when given store subpath NixOS/nix#10549

Merged

ralismark mentioned this issue Sep 7, 2024

error: path '/nix/store/...-neovim-0.10.0/bin/nvim' is not in the Nix store ralismark/nix-appimage#16

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ociTools.buildContainer erroring with "path [foo] is not in the Nix store" #316652

ociTools.buildContainer erroring with "path [foo] is not in the Nix store" #316652

tomfitzhenry commented Jun 2, 2024 •

edited

Loading

ShamrockLee commented Jun 2, 2024 •

edited

Loading

ShamrockLee commented Jun 2, 2024

aib commented Jun 2, 2024 •

edited

Loading

ShamrockLee commented Jun 2, 2024

aib commented Jun 3, 2024

aib commented Jun 3, 2024 •

edited

Loading

ShamrockLee commented Jun 3, 2024

aib commented Jun 3, 2024

ShamrockLee commented Jun 3, 2024

ShamrockLee commented Jun 3, 2024

ShamrockLee commented Jun 3, 2024

aib commented Jun 3, 2024

ociTools.buildContainer erroring with "path [foo] is not in the Nix store" #316652

ociTools.buildContainer erroring with "path [foo] is not in the Nix store" #316652

Comments

tomfitzhenry commented Jun 2, 2024 • edited Loading

Describe the bug

Steps To Reproduce

Expected behavior

Notify maintainers

ShamrockLee commented Jun 2, 2024 • edited Loading

ShamrockLee commented Jun 2, 2024

aib commented Jun 2, 2024 • edited Loading

ShamrockLee commented Jun 2, 2024

aib commented Jun 3, 2024

aib commented Jun 3, 2024 • edited Loading

ShamrockLee commented Jun 3, 2024

aib commented Jun 3, 2024

ShamrockLee commented Jun 3, 2024

ShamrockLee commented Jun 3, 2024

Legacy interface, outPath, successful build

Legacy interface, getExe, successful build

Structured interface, outPath, successful build

Structured interface, getExe, failed build

ShamrockLee commented Jun 3, 2024

aib commented Jun 3, 2024

tomfitzhenry commented Jun 2, 2024 •

edited

Loading

ShamrockLee commented Jun 2, 2024 •

edited

Loading

aib commented Jun 2, 2024 •

edited

Loading

aib commented Jun 3, 2024 •

edited

Loading

Legacy interface, `outPath`, successful build

Legacy interface, `getExe`, successful build

Structured interface, `outPath`, successful build

Structured interface, `getExe`, failed build