New TeX Live withPackages function

Development
1

I wanted to flag that texlive now implements .withPackages (as soon as the PR below hits master). It is temporarily undocumented, but it works like for other languages

texlive.withPackages (with ps; [ scheme-small collection-pstricks ])

and can be called on an existing combined scheme to add packages. It’s also easier to create non-texlive packages (just make them multi-output with special outputs names tex, texdoc…).

There is an experimental interface to add the docs too:

texlive.combined.scheme-small.__overrideTeXConfig { withDocs = true; }

It would be great if people using texlive could play with this and report any bugs, issues, feedback so that we ship 23.11 in a good state and possibly with a stable overrideTeXConfig.

https://github.com/NixOS/nixpkgs/pull/250805

15 Likes
2

It’s also easier to create non-texlive packages (just make them multi-output with special outputs names tex, texdoc…).

Could you provide a little more detail on this? I’m trying to use a 2024 version of dvisvgm with the texlive-2022 that’s available in nixpkgs 23.11. I can build dvisvgm fine on its own, but it lacks the paths to texmf.cnf and other files provided by kpathsea. I’m guessing I can use texlive.withPackages to integrate it into texlive but I can’t figure out how.

3

A small question on texlive (with no intention to be rude) : most packages of nixos seem to be fresh. Is there a particular reason for texlive to stay at 2022 ?

4

It should work with e.g. texliveInfraOnly.withPackages (_: [ dvisvgm3 ]), assuming dvisvgm3 is your updated version of dvisvgm.

However, if you start with a scheme that already contains dvisvgm, the two packages will clash. Unfortunately, overriding packages deep inside TeX Live is virtually impossible at the moment.

I was hoping to make TeX Live overrideable and ship TeX Live 2022 ‘stable’ and 2023 ‘unstable’ simultaneously before 23.11, but it didn’t work out. I guess at this point we should update nixpkgs-unstable to 2023 and work towards 2023 ‘stable’ and 2024 ‘unstable’ at the next release cycle.

Switching to 2023 is a bit trickier than usual due to ConTeXt now being outside of TeX Live, but luckily @apfelkuchen6 has worked out most build details in an old PR. I hope to resume that work in the next few weeks.

PS: keeping a stable release matters for some niche use cases, such as LaTeXML, which cannot keep up with the daily TeX Live churn.

1 Like
5

Thank you! I’ll try this and update this thread with the result.

Yes, I already had some name collision issues when I tried this (with the non-working dvisvgm3). I had to move to a more minimal scheme and add in the missing packages manually.

6

@xworld21 Quick question about texliveInfraOnly:

My texlive package configuration looks like this:

  mytex = pkgs.texliveSmall.withPackages
    (ps: with ps;
      [
        dvipng latexmk ieeetran
        fourier fouriernc bbm bbm-macros  ]); #Plus more packages, no dvisvgm here

  mydvisvgm = pkgs.texliveInfraOnly.withPackages
    (_: [ dvisvgm3 ]);

and I set

  home.packages = [ mytex mydvisvgm ];

When I try this I get an error:

error: builder for '/nix/store/l1mfzwlsznhhjqnp3ccby4lxsyl2hyx3-home-manager-path.drv' failed with exit code 25;
       last 1 log lines:
       > error: collision between `/nix/store/jhdiz5ir1zaszkh5qy22a7ygz5kh8l2f-texlive-2022-env/share/texmf/ls-R' and `/nix/store/3gbxcy9s92gavmxqm32vsahzlnxjbbn4-texlive-2022-env/share/texmf/ls-R'
       For full logs, run 'nix log /nix/store/l1mfzwlsznhhjqnp3ccby4lxsyl2hyx3-home-manager-path.drv'.
error: 1 dependencies of derivation '/nix/store/8jpz8gcvr1r0j5qvw828rp5a4k8pk1jm-home-manager-generation.drv' failed to build

Am I doing it wrong?

7

Dummy question here (I am a NixOS beginner) :

texlive.withPackages (with ps; [ scheme-small collection-pstricks ])

I see a lot of commands like this with withPackages (most of the time with python), with p:, ps;,

Where can I find a proper documentation for this and how to use it ? I search a bit but nothing really convincing.

Accordinf to this discussion, it seems that it is package related. But again, the explanation does not go further.

8

You can only install one TeX Live environment at a time, so you need to put the packages together:

mytex = pkgs.texliveSmall.withPackages (ps: with ps; [ dvipng latexmk ... dvisvgm3]);
9

I believe ps stands for “package set”. When you call texlive.withPackages, the function ps: [ ps.dvipng ] is applied to the texlive.pkgs package set.

Why do that instead of say texlive.withPackages [ texlive.pkgs.dvipng ]? Because repeating texlive twice is brittle. If you create your own overridden mytexlive with a different package set, for instance replace dvisvgm with a patched version or an entirely different TeX Live year, you’d have to replace each texlive with mytexlive. By using the ps argument you ensure that withPackages is always using the correct package set, for free.

(Or to give a better example, consider python311.withPackages vs python310.withPackages: using ps ensures that you automatically pick packages built for the requested Python version.)

Of course right now overriding texlive is as hard as it gets, but it will make more sense once we improve on that, and/or have separate texlive2022, texlive2023.

1 Like
10

That worked perfectly, thank you for the explanation(s).

11

I’m using the withPackages example here for creating a custom package, but my custom package contains a Type1 font, so I need to be able to run updmap --sys --enable Map myfont.map.

However, when running that command in the installPhase I’m getting

updmap: can’t write to /nix/store/rkchl1f1kypids82zaf49bz9j3wa61lx-texlive-2022-env/share/texmf-config/web2c/updmap.cfg: Permission denied at /nix/store/sfg2ipgijzxwgdlv0q37897pl3pcfs3a-texlive-texlive-scripts.bin-66570/bin/updmap line 1590.

What’s the right way to do this?

12

At the system level, you need to override the postBuild attribute (via the usual overrideAttrs) and add your command. Something like

(texlive.withPackages (...)).overrideAttrs (prev:
  {
    postBuild = prev.postBuild + ''
      updmap --sys --enable Map myfont.map
    '';
  })

Alternatively, if you are doing this for a single user, just run updmap --user, not updmap --sys. Just keep in mind that you’ll have to run updmap --user regularly to keep your files in sync.

But I also have good news: the refactor texlive.withPackages: replace postBuild with install-tl like script by xworld21 · Pull Request #294826 · NixOS/nixpkgs · GitHub, which I have just sent now, allows font maps from custom packages by setting passthru.fontMaps = "Map myfont.map\n" instead of messing with updmap. @ngm if you are able to try that PR, or send me your custom package privately for testing, that would be great! It would help me choose the ‘correct’ interface for font maps (i.e. should passthru.fontMaps be a string, or a list of strings, …).

13

Hey @xworld21, thanks for your work on the withPackages API, it is much clearer than the old combine version, and especially makes adding custom packages so easier.

The thing I have trouble with currently is avoiding conflicts when trying to upgrade a package. My current head heck is to try the beta of minted 3.0 (GitHub - gpoore/minted at v3-dev). I would like to replace the version of minted in texkliveFull, so I tried the following:

(texliveFull.withPackages (_: [ minted3 ]))

where minted3 refers to the according derivation fetched from the GitHub I linked (I can also upload the definition if needed). This yields to conflicts between the two version:

>error: collision between `/nix/store/4nwaqg5j4h2bx0damnvh7z0yzysigq0l-minted-2.9-tex/tex/latex/minted/minted.sty' and `/nix/store/wlgw0xznax57375sg17smn531zmdnrn5-minted-v3.0.0beta5-tex/tex/latex/minted/minted.sty'

I also tried to filter packages, as it seems to have worked with the old API:

texlive_no_minted = pkgs.texliveFull // {
          pkgs = pkgs.lib.filter
            (x: (x.pname != "minted"))
            pkgs.texliveFull.pkgs;
        };

But it appears that the new pkgs attribute is read-only, and therefore does not impact the package set.

I’d like to avoid defining my package set for now (using texliveSmall for instance), as I am prototyping my document and make frequent changes to my package list. Would you have any pointers? Thx!

14

Hi @Hippwn

The thing I have trouble with currently is avoiding conflicts when trying to upgrade a package.

This is one of the reasons we haven’t retired texlive.combine yet. We need texlive.pkgs to be overrideable and that’s quite tricky to implement, although I hope to have a reasonable solution for 24.11 (the intention is to have overrideScope; the difficulty is rearranging tlpdb & binaries in a way that makes sense and is maintainable).

If minted is a dependency of scheme-full itself, you can remove minted from its tlDeps attribute, then pass that modified package to texlive.withPackages. If the dependency comes from other packages you could try the same, and pass those packages before scheme-full (whether that works or not depends on which order the dependency resolution happens – we can tweak that, if that’s useful).

Otherwise you either go deeper and patch tlpdb.nix in a local copy of Nixpkgs, or fall back to texlive.combine which supports pkgFilter.

(In the future, there will be an overrideScope method which allows replacing individual packages.

15

Hey, thanks for your answer!

I quickly inspected texlive.scheme-full, all of its dependencies are “collections”:

> builtins.map (x: x.pname) (builtins.elemAt pkgs.texlive.scheme-full.pkgs 0).tlDeps
[ "collection-basic" "collection-bibtexextra" ... "collection-xetex" ]

If understand correctly, those act as packages that could be passed to texlive.withPackages, correct?

So once I identify where minted is, I can pass all the collection-* metapackages, including the one I modified to remove the dependency on minted, plus my version of minted, to withPackages and it should work?
Thank you for your time.

16

Yes, that should definitely work.

Now that I think about it,

texlive.withPackages (ps: [ minted3 ps.scheme-full ])

might actually work too, as long as minted3 has the same pname as the other minted (just because of how dependency resolution works, not intentionally, so consider it ‘unstable’ behaviour, if it even works).

17 
texlive.withPackages (ps: [ minted3 ps.scheme-full ])

might actually work too, as long as minted3 has the same pname as the other minted (just because of how dependency resolution works, not intentionally, so consider it ‘unstable’ behaviour, if it even works).

Just tested it and it doesn’t. I also tried the other way around (since I have no real grasp on how dependency resolution works behind the curtain) and both raise the same collision error.

I’ll try playing around with the tlDeps attribute as soon as I have some time. What would be the correct way to modify the dependencies of a collection-* meta package? .overrideAttrs ?

Thanks!

18

I thought I solved it:

minted3 = pkgs.callPackage ./nix/minted3.nix { };
texlive_no_minted = pkgs.texlive.scheme-full // {
  pkgs = pkgs.lib.filter
    (x: (x.pname != "minted"))
    pkgs.texlive.scheme-full.pkgs;
};
my_texlive = pkgs.texlive.withPackages (_: [ texlive_no_minted minted3 ]);

But the same collision still arrives, although I am now unsure why. It does not work either using teclive.combine so I’m starting to question if this way of filtering dependecies really works. It’s proposed on the wiki: Tex - NixOS Wiki

19

I am afraid the NixOS wiki is severely outdated. The .pkgs attribute stopped working like that a few years ago, in favour of tlDeps, when a new cyclic dependency in TeX live broke Nixpkgs.

The ‘officially supported’ way is to modify tlDeps of the relevant collection (a simple // { tlDeps = [ ...]; } is enough), then call texlive.withPackages on the collections you need, including the modified one. (To be completed honest, I never tested this! But if it doesn’t work, it’s a bug.) Alternatively, use the old texlive.combine interface with the pkgFilter attribute to exclude the old minted.

I’ll relax the dependency resolution this weekend to allow the simpler texlive.withPackages (ps: [ minted3 ... ]).

2 Likes
20

I am afraid the NixOS wiki is severely outdated.

Oh, good to know.

Thanks for all the details! I’ll try everything tomorrow and see if I manage to make it work. I might try to update the wiki too if it works.