Nvd: Simple Nix/NixOS version diff tool

pvonmoradi · August 26, 2021, 3:53pm

So nvds functionality has been integrated into Nix’s own CLI?
Is there a way to also show (less) the diff of the derivations of each upgrade?

khumba · August 27, 2021, 1:10am

Hi @pvonmoradi!

I didn’t know of the nix command when I built nvd (plus Nix’s command was only in Nix unstable then, not sure if that’s changed now). They’re separate, but they do largely the same thing, and present their results differently. Nix’s built-in command is nice for showing disk size changes and for having a threshold for showing a package purely based on that. Nix’s command only shows versions that changed, when there are multiple for a package, whereas nvd will always show you all versions for a package. And I’m not sure Nix’s command has any support for highlighting systemPackages.

I’m not sure if you’re asking about diffing the .drv files in the Nix store, or perhaps you mean the expressions in the .nix files coming from Nixpkgs? As mentioned higher up in the thread, nix-diff and nvd can both handle derivations. The output of both can be quite noisy, but nix-diff will show the complete set of changes. If you’re just wanting to diff the expressions, you’ll have to run a diff between versions of Nixpkgs, e.g.:

$ diff --color=auto -ru /nix/var/nix/profiles/per-user/$USER/channels-{3,4}-link/nixpkgs

.drv files can’t be traced back to the .nix files they were built from, so there’s no easy way for a tool like nvd to show “here’s this upgrade and here are the changes to the expression.”

dschrempf · September 6, 2021, 9:49am

FYI, I removed nvd from my NUR because it is available in Nixpkgs.

wmertens · December 9, 2021, 1:31pm

I can see what changed after a nixos-rebuild with nix store diff-closures $(ls -d /nix/var/nix/profiles/*|tail -2)

lom · December 11, 2021, 12:35am

I have this in my rebuild script to see the diff: ls -v1 /nix/var/nix/profiles | tail -n 2 | awk '{print "/nix/var/nix/profiles/" $0}' - | xargs nvd diff

bew · December 11, 2021, 4:19am

That how I do it, using nix store diff-closure and a helper script to have some pretty prints

github.com

bew/dotfiles/blob/69eca69fb69e7f66e400843cb6fefea9204ac2a6/justfile#L12-L26


      
          # Build the current home config WITHOUT switching to it
          build-and-diff *ARGS: (_build-only ARGS)
            #!/usr/bin/env bash
            set -euo pipefail
          
          
  # NOTE: `nix-env -q --out-path home-manager-path`'s output
            #       looks like `home-manager-path /nix/store/...`
            CURRENT_HOME_MANAGER_PATH=$(nix-env -q --out-path home-manager-path | awk '{ print $2 }')
            BUILT_HOME_MANAGER_PATH="./result/home-path"
            DIFF_FILE="./.nix-lastBuild-homeDiff.txt"
          
          
  # Helper script to nicely show:
            # * `nix store diff-closures`'s output
            # * the closure sizes (and +/- diff)
            ./bin/nix-diff-closures.sh $CURRENT_HOME_MANAGER_PATH $BUILT_HOME_MANAGER_PATH -o $DIFF_FILE

And my pretty nix-diff-closures.sh helper script:

github.com

bew/dotfiles/blob/69eca69fb69e7f66e400843cb6fefea9204ac2a6/bin/nix-diff-closures.sh

#!/usr/bin/env bash

# Dependencies:
# pkg nix (for nix commands)
# pkg moreutils (for sponge)
# pkg ansifilter

# Safer shell script with these options
# -e          : exit if a command exits with non-zero status
# -u          : exit if an expanded variable does not exist
# -o pipefail : if a command in a pipeline fail, fail the pipeline
#               (e.g this now fails: false | true)
set -euo pipefail

function get_closure_size
{
  local drv_path="$1"
  nix path-info --closure-size $drv_path | awk '{ print $2 }'
}

This file has been truncated. show original

minijackson · January 5, 2022, 2:42pm

I discovered it somewhat recently, but diff-closure on a profile path is built in (I think since Nix 2.4). After a system upgrade, I like to do:

nix profile diff-closures --profile /nix/var/nix/profiles/system

which gives me the history of added/removed/changed packages in my system. Also works for home-manager and other profiles.

drupol · May 26, 2022, 8:17am

Hi,

I’m trying to add this script on my machine. I’m using HomeManager on a flake based configuration. Can you help me enabling it ? The configuration repo is here: GitHub - drupol/nixos-x260: Contains the configuration of every home computers and the configuration of my laptop (x260) is here: https://github.com/drupol/nixos-x260/blob/e561b71f2c76fc30be4ffffa2c449af4b87213f5/flake.nix#L58

Thanks!

drupol · July 29, 2022, 9:26pm

In the end, I used this: Add activation script to report changes. · drupol/nixos-x260@bf41c10 · GitHub

It might be useful for someone else.

However, I couldn’t find a way to get only the two last generations, any clue?

uep · July 30, 2022, 1:46am

I use a variation on this:

nix store diff-closures /run/*-system

Through the power of the alphabetic order, this gives me the cumulative changes between the booted and current system generations. I don’t always reboot for every upgrade, and I use this to (among other things) see if the kernel has changed.

khumba · July 30, 2022, 2:43am

That’s clever, picking up booted-system before current-system. If you’re creating new generations without rebooting, you can also specifically match the newest two generations with something like:

<nix ... | nvd diff> $(ls -dv /nix/var/nix/profiles/system-*-link | tail -2)

using -v to sort numerically.

drupol · July 30, 2022, 5:08am

Oh nice !

I tried but it end up with:

activating the configuration...
setting up /etc...
<<< /nix/var/nix/profiles/system-36-link
>>> /nix/var/nix/profiles/system-37-link
Traceback (most recent call last):
  File "/nix/store/rzmrp4hycdp39z2swr8l26s48x4i8xbi-nvd-0.1.2/bin/nvd", line 672, in <module>
    main()
  File "/nix/store/rzmrp4hycdp39z2swr8l26s48x4i8xbi-nvd-0.1.2/bin/nvd", line 664, in main
    {
  File "/nix/store/rzmrp4hycdp39z2swr8l26s48x4i8xbi-nvd-0.1.2/bin/nvd", line 495, in run_diff
    left_manifest = PackageManifest.parse_tree((left_resolved / "sw").resolve())
  File "/nix/store/rzmrp4hycdp39z2swr8l26s48x4i8xbi-nvd-0.1.2/bin/nvd", line 231, in parse_tree
    direct_deps: List[str] = subprocess.run(
  File "/nix/store/9wa02q541sxq7372f8zv27rl57aribxj-python3-3.10.5/lib/python3.10/subprocess.py", line 501, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/nix/store/9wa02q541sxq7372f8zv27rl57aribxj-python3-3.10.5/lib/python3.10/subprocess.py", line 969, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/nix/store/9wa02q541sxq7372f8zv27rl57aribxj-python3-3.10.5/lib/python3.10/subprocess.py", line 1845, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'nix-store'
Activation script snippet 'report-changes' failed (1)

I guess this is because nix-store is not available. Do you have any idea on how to fix this ?

drupol · July 30, 2022, 5:24am

I did this and it works:

  system.activationScripts.report-changes = ''
    PATH=$PATH:${lib.makeBinPath [ pkgs.nvd pkgs.nix ]}
    nvd diff $(ls -dv /nix/var/nix/profiles/system-*-link | tail -2)
  '';

Output example:

WDYT ? Is it the best option?

afreakk · January 1, 2023, 6:23pm

  system.activationScripts.report-changes = ''
    PATH=$PATH:${lib.makeBinPath [  pkgs.nix ]}
    echo "+++++CHANGES++++++"
    nix store diff-closures $(ls -dv /nix/var/nix/profiles/system-*-link/|tail -2)
  '';

This seems to work nicely also.

mightyiam · January 2, 2023, 3:56pm

What I’d like is a single switch to turn this kind of feature on in NixOS and one in home-manager, as well.

Sandro · January 3, 2023, 4:50pm

github.com

nix-community/srvos/blob/da668bf38a1f9a446e372c79f7a98225615075d4/common/upgrade-diff.nix

# MIT Jörg Thalheim - https://github.com/Mic92/dotfiles/blob/c6cad4e57016945c4816c8ec6f0a94daaa0c3203/nixos/modules/upgrade-diff.nix
{ pkgs, ... }:
{
  system.activationScripts.diff = ''
    if [[ -e /run/current-system ]]; then
      ${pkgs.nix}/bin/nix store diff-closures /run/current-system "$systemConfig"
    fi
  '';
}

Sandro · January 3, 2023, 5:39pm

and I just created an option for that nixos/activation-script: add option to diff system on user activation by SuperSandro2000 · Pull Request #208902 · NixOS/nixpkgs · GitHub

anon39437090 · June 16, 2024, 8:19am

I’m still quite new to NIxOS but this looks quite useful. How do I active this or is this still in the works because the pull request is still open?

DamienCassou · June 17, 2024, 5:41pm

Hi @anon39437090. Welcome to the Nix ecosystem. You don’t need to wait for the PR to be merged, you should be able to add the following to your NixOS configuration:

{
  system.activationScripts.diff = ''
    if [[ -e /run/current-system ]]; then
      ${pkgs.nix}/bin/nix store diff-closures /run/current-system "$systemConfig"
    fi
  '';
}

anon39437090 · June 17, 2024, 6:12pm

@DamienCassou Thanks! I didn’t think of doing that, I just added it to my configuration. What happens when it gets merge does this option become default or would I still need to add it like how it is listed in the merge request?