Nixtamal: Fulfilling input pinning for Nix

Announcements
21

Nixtamal 1.1.0 released

  • Make changes to be compatible with Cmdliner 2.x (in NixOS staging)
  • Fix manifest file errors should provide better context
  • Fix bug where missing lockfile would error out instead of writing a new one
  • Fix TUI output with UTF-8
2 Likes
22

Nixtamal 1.1.1 released

  • Fixed bug where manifest files were generated with the old schema version (nixtamal upgrade will fix it, but not the intended UX obviously).

In 1.0.0, a schema.ml module was created to help the fact that it was easy to forget to bump the version changes… to which I missed the Manifest’s version :sweat_smile:. No worries as all places now point to current properly.

1 Like
23

I did an interview on the Full Time Nix cast by @mightyiam, if the reader here is interested a bit more on the design/history. It was the first technical cast I have ever done, but I think I told most of the story I wanted with a couple fumbles. It features @nmattia , maker/maintainer of Niv who asks some insightful questions having ‘solved’ the same problem from a different angle. The biggest thing error I noticed that we hadn’t focused so much on the “the tooling for fresh-cmd isn’t stable”, as it never was—even running nix flake update depends fully on the stateful version of nix on the system it was ran (where the lack of versioning of flake.nix makes this less safe). We sorta touched on that by the 2nd time it was brought up, but I think we all missed that this is largely a non-issue as the lockfile + lock loader are is stable for building.

Additionally, 1.1.2 is released but in a ‘weird’ state. It changes 0 functionality but properly (only partially addressed in 1.1.0) migrated up to Cmdliner 2.x, the new default package version in Nixpkgs. As such, until this percolates up to unstable, I wont be adding anything to Nixtamal since we are in a state of flux—where Nixtamal doesn’t yet build on unstable or the default branch.

6 Likes
24

Reminder: Nixtamal is expected to be broken in unstable & staging-next due to the changes the default Cmdliner version bump. Everything will be good when 1.1.2 propagates thru staging-next & then unstable (which has been slower than I had hoped).

If you want the working version now, well you are in luck as Nixtamal supports declarative patching!:

patches {
	nixtamal-1.1.2 "https://patch-diff.githubusercontent.com/raw/NixOS/nixpkgs/pull/491867.patch"
}
inputs {
	nixpkgs {
		# add patches to your setup
		patches nixtamal-1.1.2
	}
}

Once the change propagates, the patch will fail to apply… but this fine—it just means it’s time to remove this patch.

Edit: I got it cherry-picked onto master since staging had been taking days. Status: Making sure you're not a bot!. Merge already into nixos-unstable & nixpkgs-unstable.

1 Like
25

Hello, this tool looks great and I actually discovered through the Full Time Nix podcast. The issues you talked about really resonated with me.

The biggest problem I’m facing currently at work is how to deal with private repositories and fetching them using credentials from an ssh agent. Would this be something Nixtamal supports?

3 Likes
26

I would love to help you, but I don’t want to be a big support thread with back-&-forth chatter. If we find a solution, we can post it here, but would you mind sending a message via:

so I can understand the private repo/SSH issue?

2 Likes
27

Nixtamal 1.1.4 released (& 1.1.3)

1.1.3: a number of documentation fixups
1.1.4: user reported bug with old Nix APIs (<2.26) & builtins.fetchGit not supporting lfs, which is now version detected.

1 Like
28

Nixtamal 1.3.0 released (& others)

  • BREAKING: nixtamal set-up now uses --nixpkgs-channel over --nixpkgs-branch which makes the term more generic to support Git, Git archive, or Channels archive (since this is just on set up, it shouldn’t affect existing setups in any way)
  • add --use-channel if you want to fetch channels from channels.nixos.org
  • multi-output derivation (now you can opt-in/out of outputs, & it installs fine in all way but when using with nix-shell you will likely want the outputsToInstall defaults of -p nixtamal.{bin,data,doc,man}); this also means the base closure is no longer measured in gigabytes but megabytes
  • BLAKE3 hash support detection (which touches a file to avoid expensive evaluation)
  • add to ignore files on set-up (can pass a manual VCS flag, or it will try to detect)
  • fix bug in Git’s default fresh-cmd for ls-remote for branches
3 Likes
29

Nixtamal 1.5.0 released

  • New 1.1.0 schema adds name=… to set the Nix store name (will default to $INPUT-src)
  • Default fresh-cmd for File & Archive kinds using ETag or Link immutable as a canary (tho I still recommend you set manuall); this also required adding cURL & GNU AWK to the closure
  • Buffer logs until after the TUI experience to avoid the choppy behavior some may have seen
  • Error when manifest & lockfile versions aren’t latest prompting users to nixtamal upgrade
  • nixtamal check-soundness verifies that the Darcs context files exist
  • Fixed bug where nixtamal lock --force $INPUT(S) would drop unchosen values
  • nixtamal lock --force warns but proceeds on corrupted/missing lockfiles
  • open the lockfile after checking the JSON encoding, which caused files to be truncated on failur
4 Likes
30

Nixtamal 1.6.0 released

since 1.5.0:

  • Schema 1.2.0 schema adds global default-gc=… & input gc=…, while working still on experimental garbage collection support (still behind env var, not yet ready for production)
  • nixtamal upgrade reloads the manifest + lockfile to not drop previous lockfile state
  • null hashes are candidates for nixtamal refresh now regardless of freshness (this state comes up when fetching fails, the hash will be locked @ null)
  • nixtamal tweak better handles shell & whitespaces
  • store names are derived if missing from lockfile, manfest name is also prefered
  • lock loader Fossil typo
4 Likes
31

Nixtamal 1.7.0 released

  • Shell completions for Bash, fish, Zsh
32

But I use only nushell

33

Cmdliner doesn’t support nushell… or fish shell is what I use so I built the completion manually O_o

34

You could use fish as nushell completer. External Completers | Nushell

35

This question is not specifically related to Nixtamal, but I choose to ask it here since I think it may be related to the input pinner area. It could possibly also be a distinguishing feature?

Determinate Systems recently presented “Nixpkgs cooldowns”: Introducing Nixpkgs cooldowns

I understand it as way to be slightly behind the latest version to be protected if/when a recent version of some package becomes compromised

But couldn’t this feature be even better implemented by an input pinner? I am thinking of some capability to both roll forward and backward with the pinned versions. Say I want to stay 10 days behind, could an input pinner solve that?

1 Like
36

Why write any code when you can just do this (don’t do this!):

branch=nixos-unstable
until=$(date -u -d '7 days ago' '+%Y-%m-%dT%H:%M:%SZ')

rev=$(
  curl -fsSL \
    "https://api.github.com/repos/NixOS/nixpkgs/commits?sha=$branch&until=$until&per_page=1" |
  jq -er '.[0].sha'
)

nix flake lock --override-input nixpkgs "github:NixOS/nixpkgs/$rev"
3 Likes
37

fresh-cmd just executes a command so if you want to add a “cooldown” like @linus307 ’s example, you can. Since a subshell would be involved for resolving a relative date to absolute ISO, it would need to be in a script—meaning you’d build a script just like that, but return the $rev.

#!/bin/sh
# nixpkgs-rev-with-cooldown.sh
branch="nixos-unstable"
until=$(date -u -d '7 days ago' '+%Y-%m-%dT%H:%M:%SZ')

curl -fsSL "https://api.github.com/repos/NixOS/nixpkgs/commits?sha=$branch&until=$until&per_page=1"  \
	| jq -er '.[0].sha'

// manifest.kdl
inputs {
	nixpkgs {
		archive {
			url "https://github.com/NixOS/nixpkgs/archive/{{fresh_value}}.tar.gz"
		}
		hash algorithm=SHA-256
		fresh-cmd {
			$ nixpkgs-rev-with-cooldown.sh
		}
	}

Then every time you run nixtamal refresh you will get the latest + 7 days. You can add flags to your script to make it more generic, or compile a C binary out with cURL + timer baked in… You could return the entire URL back instead of just a revision… whatever your needs are.

2 Likes
38

That’s code too though…

39

Is there a simple way to use nixtamal with configuration.nix? Maybe something similar to this (which is for npins): GitHub - d-r-a-b/nixos-config_minimal-npins: A minimal example nixos-config that uses npins to track nixpkgs version in a lockfile. ¡ GitHub ? Thanks!

40

I use Nixtamal for my system configurations, but just nixos-rebuild --no-flake --file …. I could probably put a cookbook entry for it, but I use Nix to glue my setup together in a manner nixos-rebuild understands, instead of a separate script.

1 Like