How to check for nixpkgs issues before updating

Guides
1

[!Warning]
This uses gh (but can theoretically be tweaked to use the github api instead with curl and jq). Also uses sed and tr.

Background

For a long time I was updating my nixos-unstable system and hoped the update would go well or I would have wasted time and energy for nothing. (For a long time I didn’t even properly version my flake.lock…)
Recently, I decided to make a simple bash function which will get called by an alias any time before I run update my flake (using another alias).

For this to work, I keep a list of packages.txt per host, which practically is everything in environment.systemPackages = with pkgs; [...]; copy-pasted into that list. I have also added some extra package names at the bottom (cuz for example some packages are installed by enabling services and thus declared elsewhere).

In the future I could maybe plug it to a unifiedpush server and have my server notify me for new issues I may be interested in or something.

Function1 - main

The script:

  • takes the packages.txt list
    • removes any white-space character
    • joins the packages in a single line
  • feeds the list into gh which searches the nixpkgs repo and fetches issues that
    • are still open
    • are the 100 latest
    • are made in the last 30 days
    • contain these package names in their title (only)
    • are labeled with “0.kind: build failure”
    • are sorted by date-created
      I find this to be a somewhat good filter, because otherwise (especially with so many packages that I have) it’d fetch like half of the issues on the repo.

The nix-issues() function is this (the comments explain a lot):

nix-issues() - click to expand 
# nix-issues
# reads a file with a list of packages and fetches recent related issues from nixpkgs repo created in the last 30 days
nix-issues() {
	# read from ~/dotfiles/.config/nixos-config/hosts/$(hostname)/packages.txt
	# -e flag per each replacement operation
	# "s/#.*$//"	- delete comments (starting with "#" and ending till the end of the line)
	# "s/\s*//g"	- delete whitespace characters
	# "/^\s*$/d"	- delete any line and any whitespace characters it has in it (though whitespace chars are already removed)
	# "s/^/\"/"		- add " to the beginning of each line
	# "s/$/\" OR/g"	- replace the end of each line with " OR"
	# '\n' ' '		- replace each new line with a space (converts the list in to a single line)
	# "s/...$//"	- delete the last 3 chars of the line (deletes the trailing " OR")
	issueslist="$(sed -e "s/#.*$//" -e "s/\s*//g" -e "/^\s*$/d" -e "s/^/\"/" -e "s/$/\" OR/g" < ~/dotfiles/.config/nixos-config/hosts/"$(hostname)"/packages.txt | tr '\n' ' ' | sed "s/...$//")"

	# limits search to:
	# issues
	# open issues
	# issues created in the last 30 days
	# issues that contain the search terms in title
	# sorts by newest created
	issueslist+="is:issue state:open created:>$(date --date="$(date) - 30 day" +%Y-%m-%d) in:title sort:created-desc"

	# uses gh to list issues of nixpkgs repo on github issues
	# limits results to:
	# previous constraints
	# issues with the label "0.kind: build failure"
	# latest 100 that match
	gh issue list --repo https://github.com/NixOS/nixpkgs --search "$issueslist" --label="0.kind: build failure" --limit=100

	# exit program
	exit 0

	# Resources:
	# https://stackoverflow.com/questions/1521462/looping-through-the-content-of-a-file-in-bash
	# https://unix.stackexchange.com/questions/164826/date-command-iso-8601-option
	# https://stackoverflow.com/questions/18180581/subtract-days-from-a-date-in-bash
	# https://linuxsimply.com/bash-scripting-tutorial/loop/while-loop/while-loop-one-line/
	# https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests
	# https://docs.github.com/en/search-github/getting-started-with-searching-on-github/sorting-search-results
	# https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/filtering-and-searching-issues-and-pull-requests
	# https://www.geeksforgeeks.org/linux-unix/sed-command-in-linux-unix-with-examples/
	# https://stackoverflow.com/questions/16414410/delete-empty-lines-using-sed
	# https://stackoverflow.com/questions/7558557/how-to-remove-tabs-from-blank-lines-using-sed
	# https://stackoverflow.com/questions/4181703/how-to-concatenate-string-variables-in-bash
}

Function2 - relaxed

Because the previous script can be a bit too restrictive, I’ve made a “relaxed” version of it, which has to be manually called and compared to the previous one fetches issues that:

  • are both open and closed
  • are the 500 latest
  • are made in the last 60 days
  • contain these package names both in their title and body
  • contain all labels

This is the nix-issues-relaxed() function:

nix-issues-relaxed() - click to expand 
# nix-issues-relaxed
# reads a file with a list of packages and fetches recent related issues from nixpkgs repo created in the last 30 days (more than the nix-issues alias)
nix-issues-relaxed() {
	# read from ~/dotfiles/.config/nixos-config/hosts/$(hostname)/packages.txt
	# sed "1stPart/2ndPart/3rdPart/4thPart"
	# sed "s/old_word/new_word/n"	s: substitute n:parameters like d (delete), g (global), n: (nth occurance, replace with number)
	# -e flag per each replacement operation
	# "s/#.*$//"	- delete comments (starting with "#" and ending till the end of the line)
	# "s/\s*//g"	- delete whitespace characters
	# "s/^\s*$/d"	- delete any line and any whitespace characters it has in it (though whitespace chars are already removed)
	# "s/^/\"/"		- add " to the beginning of each line
	# "s/$/\" OR/g"	- replace the end of each line with " OR"
	# tr '\n' ' '	- replace (truncate) each new line with a space (converts the list in to a single line)
	# "s/...$//"	- delete the last 3 chars of the line (deletes the trailing " OR")
	issueslist="$(sed -e "s/#.*$//" -e "s/\s*//g" -e "/^\s*$/d" -e "s/^/\"/" -e "s/$/\" OR/g" < ~/dotfiles/.config/nixos-config/hosts/"$(hostname)"/packages.txt | tr '\n' ' ' | sed "s/...$//")"

	# limits search to:
	# issues
	# open and closed issues (hence state:closed)
	# issues created in the last 60 days (hence "- 60 day")
	# sorts by newest created
	# includes:
	# issues that contain the search terms both in title and body (hence no "in:title")
	issueslist+="is:issue state:open state:closed created:>$(date --date="$(date) - 60 day" +%Y-%m-%d) sort:created-desc"

	# uses gh to list issues of nixpkgs repo on github issues
	# limits results to:
	# previous constraints
	# latest 500 that match (unlike the limit of 100 in nix-issues alias)
	# includes:
	# issues with any label (hence no "--label="0.kind: build failure"")
	gh issue list --repo https://github.com/NixOS/nixpkgs --search "$issueslist" --limit=500

	# exit program
	exit 0

	# Resources:
	# https://www.geeksforgeeks.org/linux-unix/sed-command-in-linux-unix-with-examples/
	# https://stackoverflow.com/questions/1521462/looping-through-the-content-of-a-file-in-bash
	# https://unix.stackexchange.com/questions/164826/date-command-iso-8601-option
	# https://stackoverflow.com/questions/18180581/subtract-days-from-a-date-in-bash
	# https://linuxsimply.com/bash-scripting-tutorial/loop/while-loop/while-loop-one-line/
	# https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests
	# https://docs.github.com/en/search-github/getting-started-with-searching-on-github/sorting-search-results
	# https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/filtering-and-searching-issues-and-pull-requests
	# https://www.geeksforgeeks.org/linux-unix/sed-command-in-linux-unix-with-examples/
	# https://stackoverflow.com/questions/16414410/delete-empty-lines-using-sed
	# https://stackoverflow.com/questions/7558557/how-to-remove-tabs-from-blank-lines-using-sed
	# https://stackoverflow.com/questions/4181703/how-to-concatenate-string-variables-in-bash
}

Bash script

These and other functions are then packaged in a bash script bash-scripts.sh which starts with:

bash-scripts.sh - BEGIN - click to expand 
#!/usr/bin/env bash

# https://gist.github.com/akrasic/380bda362e0420be08709152c91ca1f9
# set -e option instructs bash to immediately exit if any command [1] has a non-zero exit status
# set -x enables a mode of the shell where all executed commands are printed to the terminal.
# set -u affects variables. When set, a reference to any variable you haven't previously defined - with the exceptions of * and @ - is an error, and causes the program to immediately exit
# set -o pipefail this setting prevents errors in a pipeline from being masked
set -euo pipefail

And ends with:

bash-scripts.sh - END - click to expand 
# The script contains f1, f2 and f3 functions. The variable f_call is assigned the value of the first command-line argument ($1), which should be the name of the function to be called.
# The shift command is used to shift the command-line arguments to the left, effectively discarding the first argument (the function name).
# Finally, the specified function ($f_call) is called with any remaining command-line arguments (“$@”). This allows users to pass additional arguments to the chosen function.
# https://linuxsimply.com/bash-scripting-tutorial/functions/call-function/
f_call=$1; shift; $f_call "$@"

# If the above line doesn't call any command, the following text will be displayed
echo "Wrong input!"

This script is called by different aliases which call each appropriate function stated in an aliases.nix module of mine.

Aliases

In my aliases.nix I have the following related aliases (the 3rd is a bonus, simply fetches details about a specific issues on the nixpkgs repo):

aliases - click to expand 
#reads a file with a list of packages and fetches recent related issues from nixpkgs repo created in the last 30 days
#calls nix-issues function inside ./bash-scripts.sh
#uses gh
nix-issues = "~/dotfiles/.config/nixos-config/modules/non-nix-scripts/bash-scripts.sh nix-issues";
#reads a file with a list of packages and fetches recent related issues from nixpkgs repo created in the last 30 days (more than the nix-issues alias)
#calls nix-issues function inside ./bash-scripts.sh
#uses gh
nix-issues-relaxed = "~/dotfiles/.config/nixos-config/modules/non-nix-scripts/bash-scripts.sh nix-issues-relaxed";
#shows details of an issue from the nixpkgs repo
# "s/[^0-9]//g" - removes any non-digit characters
#uses gh
nix-issues-view = "read -r -p \"Type the number of issue: \" issuenumber && issuenumber=\"$(echo \"$issuenumber\" | sed \"s/[^0-9]//g\")\" && gh issue view \"$issuenumber\" --repo https://github.com/NixOS/nixpkgs --comments && unset issuenumber";

Links

All these exist in my nixos git repo which has the dotfiles of my systems. You can them (and more) on my codeberg account:

Demo

When running

$ nix-upgrades

or

$ nix-updates

I get this dialog first:

image
image1496×854 182 KB

After viewing the issues I press q and it waits for me to press any key to continue updating/upgrading the system:
image