Flakes, allow unfree package causes error

mobergmann · September 16, 2024, 6:59pm

I don’t really understand an error I get with an flake:

I have a devShell in a nix flake, where I include the unstable channel. The flake also includes the unfree package cplex. Currently, when I want to enter the shell, I run export NIXPKGS_ALLOW_UNFREE=1 && nix develop --impure.
I would like to allow the (or just all) unfree packages within the flake, so I can just run nix develop. AFAIK I have to add let pkgs = import nixpkgs { inherit system; config.allowUnfree = true; }; to the flake (at the top, after outputs, you know which place I mean).
But when I then run nix develop, without and even with --impure, I get the following error:

➜  tef git:(enhancement/spr-information) ✗ cat flake.nix
{
  description = "A flake for providing tef and the Example as an output and an development enviroment for tef.";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
  };

  outputs = { self, nixpkgs }: 
    let
      system = "x86_64-linux";

      pkgs = import nixpkgs {
        inherit system;
        config.allowUnfree = true;
      };

    in {
      devShells.x86_64-linux.default = pkgs.mkShell {
        packages = [
          pkgs.git
          pkgs.ninja
          pkgs.cmake
          pkgs.llvmPackages.openmp
          pkgs.cplex
          pkgs.boost
          pkgs.clang-tools
          pkgs.clang # set compiler after tools, as that influences the version.
                     # See https://github.com/NixOS/nixpkgs/issues/76486 for more information
        ];
        env = { CPLEX_PATH = pkgs.cplex; };
        shellHook = ''
          alias configure="cmake -Wno-dev -B build \
                                 -D CMAKE_CXX_COMPILER=clang++ -G Ninja \
                                 -D CMAKE_EXPORT_COMPILE_COMMANDS=ON \
                                 -D CMAKE_COLOR_DIAGNOSTICS=ON \
                                 -D CPLEX_ROOT_DIR=$CPLEX_PATH \
                                 -D TEF_BUILD_EXAMPLES=ON \
                                 -D TEF_BUILD_TESTS=ON"
          alias build="cmake --build build"
          configure
          echo "Use the alias configure in the base of the project, to generate the build files."
          echo "Use the alias build in the base of the project, to compile it."
        '';
      };
    };
}
➜  tef git:(enhancement/spr-information) ✗ nix develop --impure
warning: Git tree '/home/mobergmann/Source/tef' is dirty
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/5w3dp0m37794iffsbm9vd9f1xmmhda6i-source/pkgs/stdenv/generic/make-derivation.nix:336:7

       … while evaluating attribute 'CPLEX_PATH' of derivation 'nix-shell'

         at «none»:0: (source not available)

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: This nix expression requires that the cplex installer is already
       downloaded to your machine. Get it from IBM:
       https://www.ibm.com/support/pages/downloading-ibm-ilog-cplex-optimization-studio-2211

       Set `cplex.releasePath = /path/to/download;` in your
       ~/.config/nixpkgs/config.nix for `nix-*` commands, or
       `config.cplex.releasePath = /path/to/download;` in your
       `configuration.nix` for NixOS.
➜  tef git:(enhancement/spr-information) ✗ nix develop         
warning: Git tree '/home/mobergmann/Source/tef' is dirty
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/5w3dp0m37794iffsbm9vd9f1xmmhda6i-source/pkgs/stdenv/generic/make-derivation.nix:336:7

       … while evaluating attribute 'CPLEX_PATH' of derivation 'nix-shell'

         at «none»:0: (source not available)

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: This nix expression requires that the cplex installer is already
       downloaded to your machine. Get it from IBM:
       https://www.ibm.com/support/pages/downloading-ibm-ilog-cplex-optimization-studio-2211

       Set `cplex.releasePath = /path/to/download;` in your
       ~/.config/nixpkgs/config.nix for `nix-*` commands, or
       `config.cplex.releasePath = /path/to/download;` in your
       `configuration.nix` for NixOS.
➜  tef git:(enhancement/spr-information) ✗ cat ~/.config/nixpkgs/config.nix            
{
  cplex.releasePath = /home/mobergmann/Documents/tef/cplex/cplex_studio2211.linux_x86_64.bin;
}

But I did add it! Why is that? Can someone explain me why (and tell me how to fix it)?

justinas · September 16, 2024, 9:11pm

The error is unrelated to the free/unfree licensing toggle. The reasoning for this requirement is also documented in cplex/default.nix at the top of the file. It is also a known pattern for some derivations in Nixpkgs, see docs for requireFile.

The easiest fix would be to do as the error message says (and you’ll still have to use --impure, unless you include the downloaded file in your flake’s tree, which is probably a bad idea for other reasons).

waffle8946 · September 16, 2024, 9:33pm

mobergmann:

pkgs.mkShell {
        packages = [
          pkgs.git
          pkgs.ninja
          pkgs.cmake
          pkgs.llvmPackages.openmp
          pkgs.cplex
          pkgs.boost
          pkgs.clang-tools
          pkgs.clang # set compiler after tools, as that influences the version.
                     # See https://github.com/NixOS/nixpkgs/issues/76486 for more information
        ];

Unrelated, but you should override the stdenv if you want to change the C compiler:

pkgs.mkShell.override { stdenv = pkgs.clangStdenv; } {
  packages = [
    # add stuff here
  ];
}

mobergmann · September 17, 2024, 9:21am

But I have done what the error message sad. If you look at the last command of my terminal output:

~ cat ~/.config/nixpkgs/config.nix            
{
  cplex.releasePath = /home/mobergmann/Documents/tef/cplex/cplex_studio2211.linux_x86_64.bin;
}

you can see that the file is present on my system and set in the config.nix file.
I don’t really understand why this error only occurs, when I include the unfree setting. When I remove the unfree setting (and export allow_unfree and run the flake impure) the flake loads just fine.

justinas · September 17, 2024, 11:19am

My bad for not reading the output thoroughly.

When I remove the unfree setting (and export allow_unfree and run the flake impure) the flake loads just fine.

The important part is --impure, not the presence or absence of allowUnfree in the Nix code or in the env var. You probably did:

Add --impure to the terminal command
Add NIXPKGS_ALLOW_UNFREE=1 to the terminal
Remove allowUnfree = true; from Nix code itself

all at once, hence it seems to you like allowUnfree = true; is the part that breaks things.

Without --impure, flakes can not access arbitrary paths on your machine - neither ~/.config/nixpkgs/config.nix, nor /home/mobergmann/Documents/tef/cplex/cplex_studio2211.linux_x86_64.bin. That’s part of flakes’ design. Sadly, you will need --impure as long as you need to access files outside of your flake’s root.

mobergmann · September 17, 2024, 2:01pm

Ah, okay. That answered part of my question. I thought something like that myself. But what still confuses me is, that when I do:

Add allowUnfree = true; to the flake
Set the cplex.releasePath variable in the ~/.config/nixpkgs/config.nix
Run nix develop --impure

I still get the below error, despite me using an impure environment.

➜  tef git:(enhancement/spr-information) ✗ nix develop --impure
warning: Git tree '/home/mobergmann/Source/tef' is dirty
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/5w3dp0m37794iffsbm9vd9f1xmmhda6i-source/pkgs/stdenv/generic/make-derivation.nix:336:7

       … while evaluating attribute 'CPLEX_PATH' of derivation 'nix-shell'

         at «none»:0: (source not available)

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: This nix expression requires that the cplex installer is already
       downloaded to your machine. Get it from IBM:
       https://www.ibm.com/support/pages/downloading-ibm-ilog-cplex-optimization-studio-2211

       Set `cplex.releasePath = /path/to/download;` in your
       ~/.config/nixpkgs/config.nix for `nix-*` commands, or
       `config.cplex.releasePath = /path/to/download;` in your
       `configuration.nix` for NixOS.

I would have thought, that, with an impure environment, that error in this >>configuration<< would be gone.

APPENDIX

Even setting NIXPKGS_ALLOW_UNFREE=1 while allowUnfree = true; is set in the flake and then running an impure devShell causes the error:

➜  tef git:(enhancement/spr-information) ✗ export NIXPKGS_ALLOW_UNFREE=1 
➜  tef git:(enhancement/spr-information) ✗ nix develop --impure         
warning: Git tree '/home/mobergmann/Source/tef' is dirty
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/5w3dp0m37794iffsbm9vd9f1xmmhda6i-source/pkgs/stdenv/generic/make-derivation.nix:336:7

       … while evaluating attribute 'CPLEX_PATH' of derivation 'nix-shell'

         at «none»:0: (source not available)

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: This nix expression requires that the cplex installer is already
       downloaded to your machine. Get it from IBM:
       https://www.ibm.com/support/pages/downloading-ibm-ilog-cplex-optimization-studio-2211

       Set `cplex.releasePath = /path/to/download;` in your
       ~/.config/nixpkgs/config.nix for `nix-*` commands, or
       `config.cplex.releasePath = /path/to/download;` in your
       `configuration.nix` for NixOS.

justinas · September 17, 2024, 2:14pm

It seems like config.nix will only get read if you do not explicitly pass a config when calling import nixpkgs { ... }.

github.com

NixOS/nixpkgs/blob/0e8af181c831c1ff3f232c127d21ab7cfe8a66a1/pkgs/top-level/impure.nix#L24-L34


      
          , # Fallback: The contents of the configuration file found at $NIXPKGS_CONFIG or
            # $HOME/.config/nixpkgs/config.nix.
            config ? let
                configFile = builtins.getEnv "NIXPKGS_CONFIG";
                configFile2 = homeDir + "/.config/nixpkgs/config.nix";
                configFile3 = homeDir + "/.nixpkgs/config.nix"; # obsolete
              in
                if configFile != "" && builtins.pathExists configFile then import configFile
                else if homeDir != "" && builtins.pathExists configFile2 then import configFile2
                else if homeDir != "" && builtins.pathExists configFile3 then import configFile3
                else {}

Basically, you need to choose whether you use environment variables / implicit config from config.nix, or explicitly pass a config when importing the nixpkgs. The former is much more convenient when using ad-hoc commands such as nix-shell -p some-unfree-package or nix shell nixpkgs#some-unfree-package --impure.

I do agree that it can be quite confusing how pure/impure parts of Nixpkgs interact.

mobergmann · September 17, 2024, 2:16pm

Ahhhh, Ohhhh. I see… That is… unfortunate… But that is good to know. Thanks for clarifying!