Nix function to merge attributes / records recursively and concatenate arrays

Help
1

Does someone know such function that merges list of records

  • if all values to merge are records - merge them recursively
  • if all values to merge are arrays - concatenate arrays
  • If values can’t be merged - the latter value is preferred

Example 1:

recursiveMergeAttrs [
  { a = "x"; c = "m"; list = [1]; }
  { a = "y"; b = "z"; list = [2]; }
]

returns

{ a = "y"; b = "z"; c="m"; list = [1 2] }

Example 2

recursiveMergeAttrs [
  {
    boot.loader.grub.enable = true;
    boot.loader.grub.device = "/dev/hda";
  }
  {
    boot.loader.grub.device = "";
  }
]

returns

{
  boot.loader.grub.enable = true;
  boot.loader.grub.device = "";
}

P.S.

recursiveUpdate is not working

recursiveMergeAttrs = listOfAttrsets: lib.fold (attrset: acc: lib.recursiveUpdate attrset acc) {} listOfAttrsets

recursiveMergeAttrs [ { a = "x"; c = "m"; list = [1]; } { a = "y"; b = "z"; list = [2]; } ]

returns 

{ a = "y"; b = "z"; c = "m"; list = [ 2 ]; }
2

The nixos module system does this. But you need to declare types of the fields, as on different occasions you need “different kind” of merging.

3

This would be a nice addition next to lib.recursiveUpdate
The module system, yes, requires type declaration, and in some cases leads to infinite recursion
(I recently hit i.r. trying to do

config = 
  lib.mkMerge (map (x: { networking = { .......; }; })
                   config.henet-tunnelbroker.tunnels
              );

while a simple merge function could work here, so I ended up with lib.recursiveUpdate + ad-hoc handling of arrays)

4

Has anything been added to lib since? Seems like a useful function to have to merge deeply nested attrsets with lists.

edit: I guess a PR would be the way to go if not.

5

I would really need this! I don’t have the capacities to create the PR myself right now, though. Does it make sense to create a feature request on GitHub?

6

Use lib.mkMerge. As said above, the module system handles this.

7

You might be interested in infuse, which can do this (and many other things) without dragging in the whole module system.

8 
    deepMerge = lhs: rhs:
      lhs // rhs // (builtins.mapAttrs
        (name: value:
          if (builtins.hasAttr name lhs &&
              builtins.isAttrs value &&
              builtins.isAttrs lhs.${name})
          then
            deepMerge lhs.${name} value
          else if (builtins.hasAttr name lhs &&
                   builtins.isList value &&
                   builtins.isList lhs.${name})
          then
            lhs.${name} ++ value
          else
            value)
        rhs);

there is also nix function to merge attributes / records recursively and concatenate arrays - Stack Overflow if you have nixpkgs’ lib. (note that their answer uniques lists)

9

as both let expression and attribute selection was already available from the previous stable nix version of 2.18, i would suggest modifying the function as below:

      deepMerge =
        lhs: rhs:
        lhs
        // rhs
        // (builtins.mapAttrs (
          rName: rValue:
          let
            lValue = lhs.${rName} or null;
          in
          if builtins.isAttrs lValue && builtins.isAttrs rValue then
            deepMerge lValue rValue
          else if builtins.isList lValue && builtins.isList rValue then
            lValue ++ rValue
          else
            rValue
        ) rhs);