Isn’t that the point of this discussion?
3 Likes
Now it’s getting close to stabilizing, I’m going to chuck Oils out there as a left-field option too. Bash-compatible (and in fact has uses as a bash ‘linter’) by default, with extra functionality that can be incrementally enabled, including very python-like expressions. Might be a way of writing light, low-dependency scripts without falling into some of the more horrendous bash pit-traps…
6 Likes
It’s arguably hard to end up with a runtime that entirely lacks python anyway, though, let alone (ba)sh.
I think a broad policy would be somewhat overbearing. It’d make zero sense to enforce Rust to handle the emacs modules, for example. I figure contributors know their use cases well enough, and reviewers are there to catch obviously dumb ideas.
This will naturally gravitate towards the most popular languages + bash - which I’d argue is good, since those will be most accessible, too.
Rust and Python definitely seem like pretty good contenders for the most generally accessible languages out there atm, though I’m a bit surprised there isn’t more go in nixpkgs. I guess its build system is a bit of a pain in nix contexts, but Rust isn’t that much better.
Haskell & co. may seem like a natural fit, especially given how many folks in the community are involved with it, but IME there are enough people out there who have never touched a functional language (or have and have refused to since, to the point that nix already scares them enough) that it would significantly reduce the number of people contributing, which is probably not a good thing.
6 Likes
FWIW Nixpkgs is using extremely niche bash/shell features at times and Oils sadly lacks some Bash features to make it work smoothly (biggest being let).
I’ve been tinkering with replacing Bash with Oils in the stdenv in a repo I called oily-nixpkgs and I think I’m not too far from being able to build bash with the huge bash cruft it requires.
I just had my second child and some other private things keeping me busy but if some more people would be interested in pushing this with me I’d gladly start picking it up again.
I’m sure Oils can replace at some point even Python scripts. But it’s not going to replace things which require binary operations and the like. IMO using Oils for ‘higher level’ (e.g. strings/commands as output) and Rust for lower levels is good enough.
10 Likes
Am I crazy for thinking Rust isn’t suitable for the same reason as Haskell? It’s difficult and not that popular.
Python is at least easy to pick up and much more in use.
1 Like
As was discussed in the nix-windows thread, I still think nushell is the best option I’ve heard of for trying to promote a new baseline for scripts in nixpkgs.
Particularly, it works on Windows, as well as the unixes.
I like the momentum of switch-ng, etc using Rust. But there’s a lot of last-mile scripts that don’t suit themselves to compiled languages in general; I think nushell scripts are a good fit. I’ve pretty much been authoring new scripts in nushell for over a year and am happy (despite some churn).
(To steelman my own suggestion, nushell still changes things between releases that would be annoying or worse. And it lacks trap, though I’m guessing that’s rarely used in these types of glue-y scripts)
5 Likes
At least among nixpkgs maintainers, Rust seems quite popular. And I think Haskell is significantly more problematic for people to pick up, because it has very little common ground with familiar imperative languages. I think Rust is honestly much more similar to Java or especially Swift than it is to Haskell. So I think it’s much easier to pick up, if only because of familiarity. EDIT: Then again this is Nix we’re talking about, where some of the unfamiliarity of Haskell is already present in the Nix language itself.
5 Likes
Adding to this, there is this proof of concept for using Nushell as a builder GitHub - DeterminateSystems/nuenv: A Nushell environment for Nix
1 Like
I too really like nushell from my couple of attempts using it but I see two issues with it:
- Hard to bootstrap. It needs rust which has quite a few dependencies itself and none of those could use nushell.
- It’s quite a moving target still. We need something that has a stable syntax that isn’t invalidated every few years, requiring possibly thousands of packages to change their code.
8 Likes
Even if true (which I don’t know of any survey done to confrm), that would only apply to current maintainers, and shuts out new contributors who don’t know Rust. And it’s not the kind of language you pick up overnight, I can barely keep track of how to read it sonce they keep changing the language every 3 months, much less write it.
And on that note, its instability makes it quite unsuitable as well, as rust releases often cause massive breakages in nixpkgs (see: rust 1.81). At least old python versions are kept around in nixpkgs, giving maintainers a little bit of time to catch up by pinning python temporarily, with rust you’re forced to use current rust and all tooling stays broken unless rewritten appropriately.
Java and Swift don’t, to my knowledge, break existing syntax. Java in particular is known for its rabid backwards-compat, though it’s probably unsuitable for tooling- and overhead-related reasons.
1 Like
What broke nixpkgs with 1.81? To my knowledge Rust has “editions” to prevent breaking changes to the syntax affecting legacy code.
I’ve certainly not had the experience you’ve had with Rust regarding constant change, in either case. Are you sure this isn’t from pre-2015?
Swift/Go would indeed be good alternatives. I suspect those languages see less adoption in the FOSS sphere because of their association with Apple/Google, rather than for any technical reasons.
1 Like
Going back to the initial question; I think the main issue is that bash is an inherently bad programming language and thus people understandably try to avoid it as soon as something becomes even only a bit complicated.
Arguing whether Rust is a good fit won’t solve that problem - as much as an RFC to define which languages are “allowed” would surely cause nothing more than endless discussions. The shell is still there as the “first” interface that’s being abandoned.
Replacing bash in the stdenv with something modern is IMO the only real long term solution.
And currently I only see 2 options. Nushell and Oils.
Given that unexpectedly many tools depend on /bin/sh (e.g. make), a legacy shell would still have to be there for the foreseeable future anyway.
With Oils it would be able to fulfill that role by replacing bash in place and just keeping the existing stdenv. People could switch to the new features with shopt -s ysh:all anywhere currently bash is used.
With Nushell we could have bash (or e.g. dash which is quite a bit faster) available with a new stdenv. And people can switch to the new stdenv.
If the modern shell still isn’t enough, a language like Rust can then be used, but it wouldn’t happen very often.
Both of the options are valid. I obviously favor Oils, but it’s up for dispute (and I guess which language becomes “stable” first?)
2 Likes
I too love nu; it has been my login shell for quite some time now.
With its fundamental focus on working with structured data transparently it also seems like a natural match for nix: nix at evaluation time, producing data for nu at runtime, with the potential to remove complexity in both — instead of (for example) emitting a lot of bash templated by that data in nix.
But these two issues are very real indeed. The second is well known, but the first is something I hadn’t particularly considered in this context.
2 Likes
RE: Rust
Oh I can’t miss this, here’s my uninformed rant.
As of yet Rust
- is not practically bootstrappable,
- encourages npm-style leafpad combinatorial explosions and hiding the complexity behind lock files.
- Additionally, every other Rust project appears to rely on unstable features unsuitable for packaging in Nixpkgs…
1 Like
wasn’t there an attempt to switch to oil in nixpkgs before? did you read up on that? in your opinion what currently prevents the switch?
1 Like
I think at that time there was just nobody who really actually tried? Also back then there was just oil which was the python interpreted (and thus very slow) version. Since then the project has been renamed to Oils or oils-for-unix with osh as compatible shell and ysh as shell with shopt -s ysh:all already set. And the C++ transpilation & GC have been finished.
Currently preventing are a few bashisms not implemented in Oils. Most notably let and declare -i - the later being used in the nixpkgs stdenv at various places.
Either someone has to contribute the feature declare -i (and thus probably a new “bash integer” type) to Oils or we have to remove these features from the nixpkgs codebase (and e.g. use ((x+= 1)) instead of just x+=1 where declare -i is removed).
I’ve gone through the process of debugging such issues, storing patched files “out-of-tree” in in the oily-nixpkgs repo & discussing possible fixes with the Oils maintainer various times (e.g. previously broken [[ -v myarr[element] ]] has been fixed in Oils). But I’ve not yet been able to get something compiled, though I believe I’m not too far from having found all Bash-incompatibilities.
The process of debugging templated bash snippets coming from anywhere being evald, etc. id quite time consuming 
One would think that Bash can be replaced “as the user frontend” (e.g. just the phases themselves) while the logic in the background is still Bash, but at least from what I’ve seen in the code I believe this would be quite a bit harder than just making the code Oils-compatible because there’s no clear distinction between “code” (as in the “stdenv” bash code) and “data” (as in user supplied bash code). I could be wrong, though.
It might be easier to just rewrite the ‘bash-stdenv’ code in Ysh, but I’d still be unsure how well the “surroundings” like all the compiler wrappers, etc. work. IIRC there’s a lot of magic happening when build helpers e.g. detect if a certain phase exists and act differently, etc.
7 Likes
I think nu shell would be a fantastic option for a lot of use cases in nixpkgs, moving target… yeah…
but ‘simple’ bootstrapping might be a possible event these days with the help of these projects:
Tho I am not aware of the state of these projects
thanks for the explanation - and if you end up making any progress please be sure to report back, i’m sure i’m not the only person interested in hearing about it 
1 Like
Correcting myself, it was 1.80:
The list of breakages is broken across 5+ comments, but it appears to be >100.
Note that the breaking change caused by Rust 1.80 is almost unprecedented in the history of Rust and many people inside the project are very unhappy about the way it was gone about. Indeed, newer versions of Rust have introduced a special warning for the breaking change, which goes to show how rare this kind of thing is.
It was definitely handled imperfectly and Rust will deservedly have to eat the reputational hit for stability, but 1.80 is not a good argument for Rust being cavalier about breaking changes in general, and I expect it will come up in Rust project discussion of any backwards‐incompatible changes for a long time to come. They’re far more thorough than us in general, e.g. with tools like Crater to test compiler changes against a huge swathe of the ecosystem.
11 Likes