How to override Python package (Anki)?

I’d like to bump the Anki version from 2.15.1 to whatever is the latest. I created this overlay:

final: prev:
# This doesn't work. Even though it looks like I'm doing exactly the same as
# https://github.com/flyingcircusio/fc-nixos/blob/a1c1063c4f2205fb6385fadc53083425451af5db/pkgs/overlay-python.nix
# https://discourse.nixos.org/t/overriding-not-being-kept/7818/2
# https://nixos.org/nixpkgs/manual/#how-to-override-a-python-package-using-overlays
# https://nixos.org/nixpkgs/manual/#python Talks about overridePythonAttrs
let
  sources = import ../../nix/sources.nix;

  pkgs = import sources.nixpkgs { };

  ankiOverlay = python-final: python-prev: {
    anki = python-prev.anki.overridePythonAttrs (oldAttrs: rec {
      version = "2.1.26";
      pname = oldAttrs.pname;
      src = python-prev.fetchurl {
        urls = [
          "https://apps.ankiweb.net/downloads/current/${pname}-${version}-source.tgz"
          # "https://apps.ankiweb.net/downloads/current/${name}-source.tgz"
          # "http://ankisrs.net/download/mirror/${name}.tgz"
          # "http://ankisrs.net/download/mirror/archive/${name}.tgz"
        ];
        sha256 = "0000000000000000000000000000000000000000000000000000";
      };
      # sha256-pkg = "0000000000000000000000000000000000000000000000000000";
      # rev-manual = "8f6387867ac37ef3fe9d0b986e70f898d1a49139";
      # sha256-manual = "0pm5slxn78r44ggvbksz7rv9hmlnsvn9z811r6f63dsc8vm6mfml";
    });
  };

in
{
  python = prev.python.override {
    packageOverrides = ankiOverlay;
  };

  python3 = prev.python3.override {
    packageOverrides = ankiOverlay;
  };

  python37 = prev.python37.override {
    packageOverrides = ankiOverlay;
  };

  python38 = prev.python38.override {
    packageOverrides = ankiOverlay;
  };
}

It took me a bit to figure out that overriding python packages is very different from overriding other packages. But my overlay is still not active. Anki is at 2.1.15. Even if I remove the content of prev.fetchurl { } it doesn’t complain about anything. I have this in my home.nix

  nixpkgs.overlays = [
    (import ../../programs/vscode/overlay.nix)
    (import ../../programs/anki/overlay.nix)
  ];

The vscode overlay works just fine and the

2 Likes

I’ve gotten a bit closer but it’s still not working.

The error became quite obvious when I added another overlay which didn’t work, even though it was super simple. The difference between the working overlays and the non-working ones was that the working ones were for things installed through home manager options, like programs.neovim.enable = true. The others came from a nixpkgs set which I imported from sources.nix from niv. Applying overlays in my home.nix through nixpkgs.overlays only applies to the pkgset which is “controlled by Home Manager”. If I add the overlays to

  pkgs = import sources.nixpkgs {
    overlays = [
      (import ./programs/kubectl/overlay.nix)
    ];
  };

the overlays start working… but Anki is still at the original version :frowning:

Learning how overlays work is a reasonable goal in its own right and there’s some info about it in the python section of the manual, but in this particular case it’d be nice (and simpler!) to send a normal PR upgrading the regular anki package for all to benefit from.

Hm that’s true but I’d like to at least have the current version working on my machine before creating a PR. Unfortunately I already tried various things from different sources. To be precise, I checked the info from

Nixpkgs 23.11 manual | Nix & NixOS Nixpkgs 23.11 manual | Nix & NixOS

yet I can’t make it work and it’s also a bit hard to debug since I don’t really know how.

I do know that there’s nothing from the overridden Anki in my store

$ ls /nix/store | rg anki
026x7hai05f7188ddynfgdq83xk5pdd2-anki-2.1.15
bcy1xidzminb307wbwglw916rrsafip3-anki-2.1.15-fish-completions.drv
bxac2039mdhxkjcdqvi4y204lywngqf7-anki-2.1.15.drv
by64k6hw5n4ng2jvnxvvgp63ch47lz4w-anki-manual-2.1.15.drv
f14acqzsayfy0564pyqkiipnif3w2yl1-anki-manual-2.1.15
ksbp4g8zg4ninjg9fvi7gacvsk8gawrg-anki-2.1.15-fish-completions
xj7qsf6mzhdxclz668qnnl3sq8h4iffn-anki-2.1.15-man
yxa1a4qaq0c1q04dza3qjzcz5gcn5kqx-anki-2.1.15-source.tgz.drv

Ok sorry for answering my own questions… anki is a normal CLI application and it’s therefore packaged in a simpler way. It is the library packages which are added to the package set of the different library versions. TIL reading the manual thoroughly helps.

I can override Anki like this:

final: prev:
{
  anki = prev.anki.overrideAttrs (oldAttrs: rec {
    version = "2.1.28";
    pname = oldAttrs.pname;
    src = prev.fetchurl {
      urls = [
        "https://github.com/ankitects/anki/archive/${version}.tar.gz"
        # "https://apps.ankiweb.net/downloads/current/${pname}-${version}-source.tgz"
        # "https://apps.ankiweb.net/downloads/current/${name}-source.tgz"
        # "http://ankisrs.net/download/mirror/${name}.tgz"
        # "http://ankisrs.net/download/mirror/archive/${name}.tgz"
      ];
      sha256 = "0000000000000000000000000000000000000000000000000000";
    };
    # sha256-pkg = "0000000000000000000000000000000000000000000000000000";
    # rev-manual = "8f6387867ac37ef3fe9d0b986e70f898d1a49139";
    # sha256-manual = "0pm5slxn78r44ggvbksz7rv9hmlnsvn9z811r6f63dsc8vm6mfml";
  });
}

Unfortunately this was actually the very first overlay I tried, but it didn’t work, since I was overlaying the wrong Nix pkgset… :man_facepalming:

I’ll create a PR for this tomorrow :bowing_man: and thanks

1 Like

It would be cool if the Anki package could be update from .15 to a newer release. Some of the addons available require a newer version of Anki (It’s up to 0.34 now). Not sure if what was posted works, but 0.28 would be a good start.

I have a video on adding a package to nixpkgs unstable https://www.youtube.com/watch?v=fvj8H5yUKu8

You can just substitute my creating a package with updating a package, and the steps are the same for PR creation.

It looked like updating it to a newer version required more work than just a version bump but I’ll look into it tomorrow, promise. Still have two days off.

I tried updating the package but I have 0 python knowledge and stuff breaks left and right. I guess I was able to adapt the build to the changed folder layout but the tests are all failing and I don’t know how pytest works and I just don’t have time to look into that right now :frowning:

I guess or hope that this is just busy work for someone who knows python and the only thing that needs to be updated are some paths here and there.

EDIT: Actually I’ll reach out to someone knowledgeable about Anki

the problem with anki is that they changed a lot of their project structure, and the current derivation makes a lot of assumptions about what is located where.

Not to mention that alot anki’s build process bootstraps itself through touching network

Yup. One of my colleagues is a frequent contributor to Anki though so he should be able to help. My hope is that the Nix way of doing things (use builders to fetch these things in advance) works as well and if not maybe some upstream changes can be made to make that more feasible. I’ll definitely keep working on this.

1 Like

@cideM can you ask them to add build instructions. Also, allow for an easier way to build the package while in a sandbox. The current derivation is running a lot of sed commands to remove network behavior.

For build instructions, there is: https://github.com/ankitects/anki/blob/ab6877c714f2930d7552cdd4c57e725047e7e80c/docs/new-platform.md and https://github.com/ankitects/anki/blob/ab6877c714f2930d7552cdd4c57e725047e7e80c/docs/linux.md.
The derivation probably needs a rewrite from scratch using buildBazelPackage. And maybe anki.pylib can get its own output to keep the closure of ankisyncd small…

Or should we just drop the source build in favour of anki-bin?

Yeah, if you’re up for it, go ahead.

As the maintainer of anki-bin, no. It’s a terrible, terrible hacky package that we should drop as soon as we get source builds going again.

I’m fairly positive that, contrary to their other build systems, they’ll stay with Bazel for longer than a year, so there shouldn’t be any huge packaging delays like this going forward.

1 Like

I’ll be happy to do reviews, but my bazel-knowledge is zero. Maybe we can have someone else help out here.