Nvd: Simple Nix/NixOS version diff tool

Announcements
1

Hi folks,

One thing I have been missing ever since I started using NixOS was a way to see how package versions were changing whenever I updated Nixpkgs. I wanted something like the “pretend” output from other distros’ package managers, but also something that wouldn’t produce thousands of lines of output when a mass rebuild happened.

So today I’m happy to announce nvd, a short script I’ve written that does just that. It prints out any versions that differ between two given closures. It also recognizes packages that are in environment.systemPackages and adds highlighting for them.

Output looks like this:

nvd
nvd964×392 68.8 KB

I hope this can be useful to you too. Most of the documentation is in the man page in the repo. I’m not aware of any Nix tools that provide similar output, so please let me know if there is something similar that I’ve missed.

[Edit 2024-09-29: Updated the link, since the nvd repo is moving.]

75 Likes
2

This looks awesome. Does it also work with home-manager’s generations and home.packages list of user-desired packages?

1 Like
3

Thanks. I have not tried home-manager yet so I will have to look into how home.packages is represented. You can at least pass nvd the paths to the buildEnvs and it will do the right thing. When it’s given two regular store paths, it takes the closures and then highlights direct dependencies.

There’s special handling for NixOS system configurations, where if sw exists in both arguments then their direct dependencies get highlighted instead. Maybe something similar would help with home-manager? For example if

nvd ~/.nix-profile "$(home-manager build)"

(or whatever the correct line is) doesn’t work as expected?

4

Tip for zsh users:

hash -d nix-hm=/nix/var/nix/profiles/per-user/$USER/home-manager
hash -d nix-now=/run/current-system
hash -d nix-boot=/nix/var/nix/profiles/system

With that you can do nvd ~nix-boot ~nix-now and nvd ~nix-hm "$(home-manager build)".

8 Likes
5

What would be the paths for use with nixops?

(edit: found them, you can use nixops dump-nix-paths, but per-machine diff would be nice)

6

It works well with home-manager, and I even integrate it in my change-reporting module (initially written for GitHub - Gabriella439/nix-diff: Explain why two Nix derivations differ, but I disabled it because it was too verbose).

{ config, pkgs, lib, ... }:

with lib;
let
  dag = config.lib.dag;
  nvd = import (pkgs.fetchFromGitLab {
    owner = "khumba";
    repo = "nvd";
    rev = "7cdaa6d818119bd7a51930d990fded5d594c6623";
    sha256 = "sha256-dQcfoMtRGg+SRvgY9pbSqlyeTozlHp3qE70egSEiFX0=";
  }) { inherit pkgs; };
in {
  options.home.report-changes.enable = mkEnableOption "report-changes";
  config = mkIf config.home.report-changes.enable {
    home.activation.report-changes = dag.entryAnywhere ''
      ${nvd}/bin/nvd $oldGenPath $newGenPath
    '';
  };
}
3 Likes
7

Thank you very much @madjar and @khumba! I now get this:

$ hm switch
[...]
<<< /nix/store/0acigs189nanzr166brbbkgf2ail3mza-home-manager-generation
>>> /nix/store/04px8q82ppnsiyxwnv2fh6h132hv3gy7-home-manager-generation
Version changes:
[U.]  #1  syncthing  1.13.1 -> 1.15.1
Closure size: 1406 -> 1406 (8 paths added, 8 paths removed, delta +0).

This is awesome!

2 Likes
8

nix-diff provides similar functionality, it’s very useful. This looks much nicer especially for things like system generations, although maybe it looks less nice on arbitrary derivations.

Being greedy… It seems to me like from a UX perspective it would be nice to have both the nix-diff and nvd functionality as output options of one derivation-diffing tool. Have you thought at all about whether you could consider merging them?

9

The main difference between nvd and nix-diff is not the output format, I think: it is that nvd works on store paths and nix-diff works on derivations. This makes it natural for nvd to answer the question “what user-visible things changed” and for nix-diff to answer “what causes this derivation to be different”.

On the other hand, I seem to remember the mechanisms are similar for both closures and derivations (in both cases, you call nix-store --query --requisites), so doing both might actually be doable.

3 Likes
10

Thanks @madjar !

In my case (just nix + home-manager on a non-NixOS, report-switching not needed) it boiled down to:

{ config, pkgs, lib, ... }:

with lib;
let
  dag = config.lib.dag;
  nvd = import (pkgs.fetchFromGitLab {
    owner = "khumba";
    repo = "nvd";
    rev = "7cdaa6d818119bd7a51930d990fded5d594c6623";
    sha256 = "sha256-dQcfoMtRGg+SRvgY9pbSqlyeTozlHp3qE70egSEiFX0=";
  }) { inherit pkgs; };
in {
  home.activation.report-changes = dag.entryAnywhere ''
    ${nvd}/bin/nvd $oldGenPath $newGenPath
  '';
}

Awesome.

5 Likes
11

Thanks for this package! I added it to my NUR repository. Since you are the maintainer, @khumba please let me know if you do not want this, see https://github.com/dschrempf/nur-packages/blob/e53af6987b40f5e0e135df2be8e0df576e92f0d1/default.nix

At the moment, I use a simple bash script to show the changes after a switch:

nixos_old_gen=$(readlink -f /run/current-system)
sudo nixos-rebuild switch
nixos_new_gen=$(readlink -f /run/current-system)
nvd "$nixos_old_gen" "$nixos_new_gen"

Then, I get the following output:

<<< /nix/store/qjgcdcz5rp74axdbi80yk2sbx82p848p-nixos-system-schwarzbaer-21.05pre282432.311ceed827f
>>> /nix/store/1vv7nyjifzhxxbq0raxhv581zxgqf480-nixos-system-schwarzbaer-21.05pre282669.e019872af81
Version changes:
[C.]  #1  busybox                   1.32.1, 1.32.1-x86_64-unknown-linux-musl -> 1.32.1
[U*]  #2  cpupower                  5.10.27 -> 5.10.29
[U.]  #3  initrd-linux              5.10.27 -> 5.10.29
[U.]  #4  linux                     5.10.27, 5.10.27-modules-shrunk -> 5.10.29, 5.10.29-modules-shrunk
[U.]  #5  nixos-system-schwarzbaer  21.05pre282432.311ceed827f -> 21.05pre282669.e019872af81
[U.]  #6  tp_smapi                  0.43-5.10.27 -> 0.43-5.10.29
[U.]  #7  x86_energy_perf_policy    5.10.27 -> 5.10.29
Added packages:
[A.]  #1  busybox-static  1.32.1-x86_64-unknown-linux-musl
Closure size: 1043 -> 1043 (49 paths added, 49 paths removed, delta +0).

Is it expected that the nixos-system-schwarzbaer is not separated into smaller chunks? Or should I use different profile paths?

Thanks!

EDIT: The NUR package failed to build, so I remove it for now, not sure why this is the case:

Run nix-env -f . -qa \* --meta --xml \
error: cannot import '/nix/store/xyg1nj013dafwcfg2z472xflxqm245yb-source', since path '/nix/store/ixka5r0lqqrf25j9fafxf2zd1c5ywmhn-source.drv' is not valid, at /home/runner/work/nur-packages/nur-packages/default.nix:30:9
Error: Process completed with exit code 1.
1 Like
12

@madjar, that’s interesting, I hadn’t thought about calling this post-activation, but that’s a really good idea.

And right, there’s nix-diff which I should have mentioned. That’s definitely the tool for finding out why results differ. It’s always a bit of work to get a good picture of the difference between two store paths. nix-diff does a really good job of showing you everything, but you have to navigate the output. If only one config file deep in the dependency tree changed, you have to think “okay I know all these hashes only changed for this reason,” since Nix has no idea about the meaning of the change. Until we can teach Nix that… nvd simplifies things by parsing names in store paths and grouping things under those names, but it can’t give the full picture – e.g. why there are two versions of a package, do they differ in build flags or architecture, or maybe it’s an entirely different package that simply shares the name. If I could have three Nix wishes, I’d ask for additional metadata on built packages to make the UI so that these things would be easier to present to the user.

nvd used to be callable with derivations, but the result isn’t pretty or usable. Lots of things in derivation closures don’t have nice names and parsing them breaks down, e.g. with source directories, or many cases of <long hex string>.patch or CVE-XXXX-XXXXX.patch. (Used to: it looks like I broke it, I’ll have to fix that. nvd should work if given a derivation or non-directory store path. It does just call nix-store -qR under the hood. Right now, the code’s expecting it to be a directory though.)

Though @michaelpj I agree that on the UI front, having a combined tool would be nice.

@dschrempf, that’s fine by me if you want to package it. Thanks! I don’t have a NUR repository set up for myself. If you’re asking about the parsing into nixos-system-schwarzbaer and 21.05pre282432.311ceed827f, that’s as correct as it can be, following how builtins.parseDrvName works. If you want to cut out the system name and the kernel and modules, you could perhaps pass /run/current-system/sw instead.

I’ve been running nvd on an SSD so far and ran it on a hard drive for the first time today. Crawling through the system-path derivation can sure take a while. I originally had a NixOS module to produce a manifest file based on environment.systemPackages, and nvd would read that file rather than walking the filesystem. I’d be open to restoring that functionality if there’s interest. (Edit: Or for home-manager of course. The idea is just to take the list of explicitly selected packages and write all their store paths to a file at a well-known path for nvd to find.)

1 Like
13

What about adding nvd to nixpkgs?

1 Like
14

That would be best. I will submit a PR. There are a couple of quick improvements I want to make first (one of them being getting rid of that directory crawl). I hadn’t seen the issues you filed @DamienCassou, apologies, when I created the GitLab repo it didn’t automatically set me to be notified on activity.

2 Likes
15

I just learned about nix store diff-closures ~nix-hm ./result that gives somewhat similar output:

<<< /nix/var/nix/profiles/per-user/nikolay/home-manager
>>> result
Version changes:
[D.]  #1  hledger           1.21 -> 1.19.1
[D.]  #2  hledger-interest  1.6.1 -> 1.6.0
[D.]  #3  hledger-ui        1.21 -> 1.19.1
Added packages:
[A.]  #1  nodejs-slim  14.16.0
Closure size: 924 -> 925 (9 paths added, 8 paths removed, delta +1).

vs

hledger: 1.21 → 1.19.1, +47.1 KiB
hledger-interest: 1.6.1 → 1.6.0, +754.7 KiB
hledger-ui: 1.21 → 1.19.1, +576.1 KiB
nodejs-slim: ∅ → 14.16.0, +35859.4 KiB
6 Likes
16

It turns out that evaluation the Nix option restrict-eval is enabled for builds in NURs, that’s why the nvd build worked locally, but failed when I uploaded the change to NUR. I guess we have to wait for the nixpkgs pull request.

@ony are you using a new version of nix? nix 2.3.10 does not have a store command (at least I cannot find it).

17 
$ nix --version
nix (Nix) 2.4pre20210326_dd77f71

I have nix store diff-closures

1 Like
18

you may have to turn on experimental-features = nix-command?

19

That looks really cool @ony, thanks for the tip! I’ve added this to my config:

  system.activationScripts.diff = ''
    ${pkgs.nixUnstable}/bin/nix store \
        --experimental-features 'nix-command' \
        diff-closures /run/current-system "$systemConfig"
  '';

Works like a charm even on the stable nix :slight_smile:

11 Likes
20

I know it’s an old topic, sorry for the gruft: But in case someone tried this my example had to be (at least as of nvd version 0.1.2):

{ config, pkgs, lib, ... }:

with lib;
let
  dag = config.lib.dag;
  nvd = import (pkgs.fetchFromGitLab {
    owner = "khumba";
    repo = "nvd";
    rev = "13d3ab1255e0de03693cecb7da9764c9afd5d472";
    sha256 = "1537s7j0m0hkahf0s1ai7bm94xj9fz6b9x78py0dn3cgnl9bfzla";
  }) { inherit pkgs; };
in {
  home.activation.report-changes = dag.entryAnywhere ''
    ${nvd}/bin/nvd diff $oldGenPath $newGenPath
  '';
}
2 Likes