Best practices for avoiding back compat breaking changes in package versions

1

I’m interested in using nix with bazel to simplify our onboarding.
It looks like a major version bump was performed on bazel at some point without much fanfare (from 0.29 to 1.0).
If this had happened and we were using nix to manage the dep it would afaict forced us to deal with some complicated back compat version break issues synchronously, which is a really tough sell.

In cases like this where back compat is broken, shouldnt a new package name be used?

If not and back compat breaking changes can be expected to sneak in easily, what are the best practices for limiting the impact of these silent version upgrades?

2

Though in another point of view, it might be better to locally maintain a custom nixpkgs and cherry-pick the required package upgrades/changes.

On the other hand, as long as it was just a simple version bump, you can use an overlay to pin the version, or to prevent unexpected changes just copy the whole derivation over into the overlay.

3

Thanks for the reply!

Are there any disadvantages to this?
Im experimenting - I think - with something like this.

Seems like pinned/overridden deps no longer benefit from binary cache so that my pinned deps seem to be getting rebuilt?

Do you know if there is a way around this?

4

you can start a cachix and push your binary cache there so you can use it anytime
or you can have CI push it automatically for you

5

Besides using a binary cache as @ihebchagra suggested, you could also pin to a specific nixpkgs revision for bazel. You could put something along the lines of the following (untested) in an overlay:

bazel = let
  nixpkgs-bazel = import (super.fetchFromGitHub {
    owner = "NixOS";
    repo = "nixpkgs";
    rev = "the revision you want to pin";
    sha256 = "the hash of this revision";
  }) {};
in nixpkgs-bazel.bazel;

Since you are using the upstream derivation, this will hit the cache.

1 Like
6

Ohh this is very interesting. Thanks, will give it a shot.

One last q - when i pin a version like this, does it also pull in dependencies from the same pinned channel?

7

nix derivations will know of all dependencies that it was built with. If there was a cache hit with bazel, then you will get all of it’s dependencies “for free”. However, if you select a commit that never had a hydra job ran on it, then you might spend a lot of time building the package set.

That said, all builds using the pinned bazel version will likely need to be built again.

For bazel specifically, it’s probably better to pull out the buildBazelPackage helper and pass that to the package you need

8

However, if you select a commit that never had a hydra job ran on it, then you might spend a lot of time building the package set.

This seems to imply that cached builds are not garbage collected - is that the case? Or if they are, any idea what the garbage collection policy is?

That said, all builds using the pinned bazel version will likely need to be built again.

Why is that @jonringer? Are you saying bazel has never been built with hydra prior to the latest version?

9

This seems to imply that cached builds are not garbage collected - is that the case? Or if they are, any idea what the garbage collection policy is?

There isn’t garbage collection from a channels standpoint, if your derivation matches an existing one exactly, you will get a cache hit. Trick is getting it to match exactly.

Locally you can garbage collect using nix-collect-garbage

That said, all builds using the pinned bazel version will likely need to be built again.

Why is that @jonringer? Are you saying bazel has never been built with hydra prior to the latest version?

bazel itself will likely be cached as you’re pulling from master. However, since the packages your building now have dependencies which differ in both the pinned repo, and the current repo, you will get new derivations which don’t exist in hydra, thus you will need to build them.