What is Nixpkgs preferred programming language?

Development Documentation
41

This has happened once.

9 Likes
42

After a quick skim of this thread, I want to voice my 2 cents:

  • This is an endless debate that’s mostly agnostic to nixpkgs.
  • An RFC to standardise a preferred language for this is fundamentally flawed IMO because you’ll just lose out on contributors who don’t know that language and don’t care or have enough time to spare to learn it – irrespective of how popular the language may be.
    • At this point we should ask ourselves if the benefit from standardisation and the language itself is worth losing out on potential contributors.

There are many people in this thread who have contributed far more than me and for far longer; my intention is to just callout saying “hey, be aware of these issues.” and not as any means to discourage the on-going discussion :smiley:

12 Likes
43

I am also curious about this, I think Go is a good middle term between Python and Rust, considering that the language is compiled and has a type system (that it is not amazing by any means, but still better than not having one), but is also more accessible than Rust and have a better standard library (though also quirk, flag for example is awful for any serious CLI tool).

If Go was a language that was more used I would probably have started nixos-rebuild-ng in Go instead of Python to have lower startup times (since I think this is an important metric for a CLI tool), though I am quite happy with the startup times right now.

5 Likes
44

Just want to note google is blocked in china, is goproxy.cn considered safe? I don’t want to hinder our Chinese contributors of which there are many.

1 Like
45

I think it’s fine according to faq #2

Edit: also see another related thread Sops-nix: Another China GFW Problem

46

You don’t need to think about this, most of the contributors you can see on GitHub have VPN enabled. Circumventing the wall is a basic skill for Chinese programmers, which is even more basic than installing Python. And after all, we are already on GitHub which is partially restricted (some functionalities are banned, others cannot establish a stable connection), and we are mostly fine with that.

3 Likes
47

Thanks for confirming, I only asked since I saw questions recently from users in China struggling to use go (e.g. Sops-nix: Another China GFW Problem). Good to know it’s not a major issue, I guess the replies are also suggesting the same thing.

1 Like
48

I just want to say that it is irresponsible to suggest that “you can just use a VPN, it’s easy,” as a solution to this problem. If you’re running nix only for your own personal jollies, fine, but if you’re trying to use it in an enterprise situation, that’s nonsense, and no, they don’t all just have VPNs.

VPNs are often illegal in China. “Use a VPN” is not a solution. I mean this with utmost respect, I’m just saying.

1 Like
49

By the way, I had a few reasons why I wrote nixos-rebuild-ng in Python that I think are related to this topic:

  • Python is a language that is already used by other tools (e.g.: systemd-boot-builder.py), so it is highly likely that any standard installation of NixOS will pull Python as a dependency anyway
    • For the few cases that it will not (e.g.: container images), I don’t except people will want nixos-rebuild in it anyway
  • Python has a good standard library (e.g.: there is a json library already that is pretty good, argparse to create CLI commands, subprocess to run other executables, etc.), that means it is less likely that I will need to pull third party dependencies, reducing the risk of breakages
    • Right now nixos-rebuild-ng has only one external dependency in tabulate, that technically is completely optional (I didn’t made it optional yet because lack of demand) and is only used by one specific command, and I don’t expect to need to add any other dependency until the end of the project
  • Good tooling: mypy and ruff are great to ensure that the code is correct (types and linting) and also ensuring that the code is formatted consistently (ruff format or black)
  • Reasonable easy to debug thanks to breakpoint()
1 Like
50

(sidenote, I think most of us nixos systemd maintainers regret that this is written in python and wish it was rust. It’s too insignificant to be worth changing though because at least python is decent)

1 Like
51

I am curious if they regret because they prefer Rust and want everything written on it or because Python was a bad choice (but not so bad that needs up a rewrite).

A write-up would be interesting by saying why, not what.

2 Likes
52

It is admittedly probably more a matter of preference than anything else, but we’ve definitely had little issues. For one, the script is a little slow, and not all of that slowness is due to unnecessary file io. And we’ve definitely had bugs caused by our use of regex in there, which also would have been bugs if we’d used regex in rust but I think python encourages regex more than rust and regex probably wouldn’t have been used.

2 Likes
53

I found it unlikely that a script so small would be CPU bound, but I never took a close look at the script so I don’t know.

Yes, this is definitely something that Python has as disadvantage against more modern languages. I kinda miss from Scala for example the string uninterpolation:

val s"jdbc:$dialect:$rest" = "jdbc:mysql:whatever"
1 Like
54

If the future of nix is tvix, then it is best to write everything else in rust atleast to keep it all in sync.

I have never used it, and not planning to anytime soon but I might have to, as it is the #1 language anyone ever talks about for the past few years.

I personally cannot bear the rust compile times, cpu usage when compiling.
So go, python are what I prefer for personal projects, depending on the project.

2 Likes
55

The future of nix is nix, hopefully :smile:

And the language of nix’s backend should have no bearing on nixpkgs.

5 Likes
56

The biggest practical downside to Rust in Nixpkgs, imo, is how long it takes for rustc to build when not cached. This often hits me pretty hard when doing bisects on Nixpkgs where Rust is touched: often, this requires both LLVM and rustc to be compiled, which takes ages. Otherwise, it doesn’t really feel like it has that many downsides versus other compiled languages.

I don’t know if that’s a situation that can be improved much, though, and it seems like a cost we’re just going to have to deal with as Rust continues to eat the world. (Thank goodness CPU manufacturers have been able to continue to push compilation times down.)

1 Like
57

@jchw Won’t that be a problem for most languages? Building python takes forever too. Most languages main implementations are rather massive builds.

Btw if you’re having trouble with caching while bisecting nixpkgs, use git bisect start --first-parent. That will only descend down the actual merge commits on master, so you won’t find yourself in the middle any branches, most importantly staging.

1 Like
58

Does building CPython really take as long as LLVM+rustc? I have a fairly high end computer with a moderate overclock and LLVM+rustc is one of the few things that can still take a half hour to build. I’m pretty sure CPython just takes a few minutes but maybe I am completely misremembering.

Sometimes I have no choice but to dig into staging because that is sometimes where the problem is. However, I greatly appreciate this advice as it will likely save me a lot of time, as I imagine it will enable narrowing down which staging merge pretty quickly.

59

Would certainly be interesting to measure on good hardware. I can imagine CPython being a few minutes on a very modern, beefy machine, but I’d have guesstimated that LLVM+rustc wouldn’t be much worse than 2-3x the duration.

I suspect that most of us aren’t working on workstations specifically designed for software builds anyway, though, so for the majority of the community most of the core language implementations will be comfortably over 20 minutes and way beyond maintaining flow state.

I should get a better CPU, CPython in 5 minutes seems incredibly useful…

60

Bah, I think I’m wrong, or at least wrong in any way that matters. If you change the Python derivation it has the knock-on effect of causing tons of other things to recompile, so irrespective of how long CPython itself takes to compile, it will still take a very long time to build CPython. I’m not sure how I came to any other conclusion (perhaps it’s a result of experiencing rebuilds for the non-default CPython version only?)

Anyway, sorry for dragging this off-topic. I suppose this strongly suggests that between CPython and Rust there is no significant advantage either way. (The calculus is probably different for some languages, e.g. I don’t think the same thing will be true of Go. But that said, if a bunch of stuff starts depending on Go like it does CPython, it won’t be that much different.)

edit: For the curious, after making a tiny change to the Python derivation, here’s how long it took to rebuild CPython on my main machine:

nix build .#python3  3.60s user 5.17s system 0% cpu 25:18.14 total

Not great. But, changing the Python derivation also causes other things to rebuild, like Perl and GCC, so it is not too surprising. I don’t know if there’s much that can be done about that :slight_smile:

3 Likes