How to avoid `with` but also duplication of code?

Help
1

I have been using inherit to avoid with (taken from this answer in The papercut thread),

let
  nixpkgs = import <nixpkgs> {};
in {
  example = let
    inherit (nixpkgs.lib) mkOption mkEnableOption;
  in [mkOption mkEnableOption];
}

but with lots of packages and the complexity of other code surrounding it, adding extra package(s) will get error-prone quickly. So I did this:

let
  # ...

  neededPkgs =
    {
      inherit (pinnedPkgs)
        entr  findutils gnumake git gnum4
        # etc.
      ;
    }
  ;

  attrSetToList =
    attrSet:
      map
        (key: builtins.getAttr key attrSet)
        (builtins.attrNames attrSet)
  ;
in
  pinnedPkgs.mkShell {
    buildInputs =
         [
           # other stuff here if needed
         ]
      ++ attrSetToList neededPkgs
    ;
  }

Am I re-inventing the wheel?

(As far as I can tell, there is still no built-in function to convert an attribute set to a list so I used the one found in [Nix-dev] Nix language: converting an attribute set to a list thread).

1 Like
2

There is builtins.attrValues which does exactly that.

1 Like
3

There is not much point in avoiding tightly scoped with IMO.

There is absolutely nothing wrong with:

{
  buildInputs = with pkgs; [ ... ];
}

It’s 100% clear where those attrs come from and if you need to differentiate locally, simply make multiple lists with local withs (or without) and concatenate them like this:

{
  buildInputs = let
    different = with differentSet; [ ... ];
    regular = with pkgs; [ ... ];
    other = [ name1 name2 ];
  in different ++ regular ++ other;
}

(Or use parenthesis to limit the scope.)
This approach is actually a lot more clear and declarative IMO.

with only gets problematic when it’s used on a larger scope, i.e. at the top of a huge package or module declaration because then you’re a lot more likely to run into namespace collisions and uncertainty.

2 Likes
4

Good point, but a constant fear of mine is that I accidentally update the wrong with-scoped list (or someone else, if it is a shared shell.nix for example), then I will have to start hunting down the culprit. In version using inherit this can never happen, but I have to admit that your example is aesthetic.

Would you expand on “Or use parenthesis to limit the scope.”?

5

Here’s an intentionally broken snippet to show off what I mean:

let
  oneSet.attr = 1;
  otherSet.attr = 2;
in

{
  buildInputs = (with oneSet; [ attr ]) ++
                (with otherSet; [ attr ]) ++
                [ attr ];
}

The with statements only apply within their parens which means the attr in the 2nd sublist is otherSet.attr and the the attr in the 3rd sublist is an undefined variable and throws an error.

This is technically less declarative since the sublists don’t have a name that describes them but this can be useful if you just need ad-hoc declarations when it’s obvious what’s what.

1 Like
6

I just learned a new thing regarding the parenthesis, thanks!

My fear unfortunately remains that (1) global variables will mask local ones, and (2) one can always introduce an extra variable that will have to be tracked down. For example:

x =
let 
  attr = 7;
  anotherAttr = 27; 
in let 
  oneSet = { attr = 1; }; 
  otherSet = { attr = 2; }; 
in 
  {
    buildInputs = 
         (with oneSet; [ attr anotherAttr ]) 
      ++ (with otherSet; [ attr ]) 
      ++ [ attr ];
  }

# x.buildInputs => [ 7 27 7 7 ]

The version with inherit is almost ugly, but extra stuff can’t sneak in.

let
  as = { a = 1; b = 2; c = 3; };
in 
{ inherit (as) a b; }
#=> { a = 1; b = 2; }

let
  as = { a = 1; b = 2; c = 3; };
in
  { inherit (as) a b d; }
#=> error: attribute 'd' missing, at (string):1:40

Seems to me that with would require to remember conventions in order to avoid getting into trouble (and I can’t even remember my sets of rules…) but inherit will slap me on the wrists if I tried to do something funny.

I should also note here that I haven’t even created a single package with Nix, so if I’m overlooking something, please let me know!

7

My fear unfortunately remains that (1) global variables will mask local ones

That is true unfortunately.

I haven’t even created a single package with Nix, so if I’m overlooking something, please let me know!

The only thing you’re overlooking is how frequently this happens in practice. I’ve never seen a namespace collision like this in Nixpkgs and they should be trivial to see coming and avoid.

1 Like
8

Have you considered using the nixpkgs lib function attrVals? Like so:

buildInputs = lib.attrVals
  [
    "entr" "findutils" "gnumake" "git" "gnum4"
    # etc
  ] pinnedPkgs

This is almost as compact as using with, but avoids its problems. The main additional cost is the quotes.

1 Like
9

No, because I just learned it from you:) Thank you!

Notes to self

Took lib.attrVals for a spin, and the inherit version reports errors at the right location whereas lib.attrVals seems to be harder to troubleshoot.

For example:

nix-repl> pkgs.lib.attrVals [ "entr" "gnumake" "cowsay" "fake-pkg"] pkgs     
[ «derivation /nix/store/...-entr-4.6.drv»
  «derivation /nix/store/...-gnumake-4.3.drv»
  error: attribute 'fake-pkg' missing, at .../lib/attrsets.nix:84:37
                                                               ^^^^^

nix-repl> let                                                        
            as = { inherit (pkgs) entr gnumake cowsay fake-pkg; };       
            y  = builtins.attrValues as;                             
          in
            y
[ «derivation /nix/store/...-cowsay-3.03+dfsg2.drv»
  «derivation /nix/store/...-entr-4.6.drv»
  error: attribute 'fake-pkg' missing, at (string):2:9
                                                   ^^^

With a nix-shell example (lines below are the content of my-shell.nix):

{ pkgs ? import <nixpkgs> {} }:

let
  neededPkgs =
    pkgs.lib.attrVals
      [ "entr" "mkdocs" "findutils" "gnumake" "git" "gnum4"
        "fake-pkg"
      ]
      pkgs
  ;
in
  pkgs.mkShell
    {
      buildInputs =
           [ /* extra stuff here */ ]
        ++ neededPkgs
      ;
    }

Output never mentions my-shell.nix, even with --show-trace:

$ nix-shell my-shell.nix
-------
error: attribute 'fake-pkg' missing, at .../lib/attrsets.nix:84:37
(use '--show-trace' to show detailed location information)

$ nix-shell --show-trace my-shell.nix
-------
error: while evaluating the attribute 'buildInputs' of the derivation 'nix-shell' at /nix/store/rvpdr9qywd7fsz624a5c255b5w5frpyd-nixos-20.09.3346.4d0ee90c6e2/nixos/pkgs/build-support/mkshell/default.nix:28:3:
while evaluating 'getOutput' at /nix/store/rvpdr9qywd7fsz624a5c255b5w5frpyd-nixos-20.09.3346.4d0ee90c6e2/nixos/lib/attrsets.nix:464:23, called from undefined position:
while evaluating anonymous function at /nix/store/rvpdr9qywd7fsz624a5c255b5w5frpyd-nixos-20.09.3346.4d0ee90c6e2/nixos/pkgs/stdenv/generic/make-derivation.nix:143:17, called from undefined position:
while evaluating anonymous function at /nix/store/rvpdr9qywd7fsz624a5c255b5w5frpyd-nixos-20.09.3346.4d0ee90c6e2/nixos/lib/attrsets.nix:84:34, called from undefined position:
attribute 'fake-pkg' missing, at /nix/store/rvpdr9qywd7fsz624a5c255b5w5frpyd-nixos-20.09.3346.4d0ee90c6e2/nixos/lib/attrsets.nix:84:37

whereas with the version using inherit:

/* 1*/ { pkgs ? import <nixpkgs> {} }:
/* 2*/  
/* 3*/ let
/* 4*/   neededPkgsAttrSet =
/* 5*/     {
/* 6*/       inherit
/* 7*/         (pkgs)
/* 8*/         entr mkdocs findutils gnumake git gnum4
/* 9*/         fake-pkg
/*10*/       ;
/*11*/     }
/*12*/   ;
/*13*/  
/*14*/   neededPkgs =
/*15*/     builtins.attrValues neededPkgsAttrSet
/*16*/   ;
/*17*/ in
/*18*/ pkgs.mkShell {
/*19*/   buildInputs =
/*20*/        [ /* extra stuff here */ ]
/*21*/     ++ neededPkgs
/*22*/   ;
/*23*/ }

Output:

$ nix-shell my-shell.nix
-------
error: attribute 'fake-pkg' missing, at ../my-shell.nix:5:13
(use '--show-trace' to show detailed location information)
1 Like