Even leaving Windows aside, there is a need for a scripting language to replace Bash in builtins.exec (why? 1. bash alone is not enough, bash does almost nothing except running other program, which have to be build somehow … so it quickly evolves to maintaining another “bootstraping” copy of <nixpkgs> for programs to run in builtins.exec 2. safety. builtins.exec code runs under user account and can do weird things, Nix own scripting engine could limit what paths the code can access, etc. Guix can happily use Scheme everywhere, but we have to use Bash every time when we need imperative code). As an embedded scripting language, it has to use the I/O functions used by Nix and therefore works in Windows if Nix is running there. And it gives us additional options to choose the language: there are much more embedded languages than standalone ones. But here 1:1 type matching with Nix would be desirable, and it is the weak point of JavaScript which lacks Int64.
Just to note: although Ruby does not support Windows well, Chef made the work to support UTF-16 strings, long file names and symlinks: https://github.com/chef/chef/tree/main/lib/chef/win32
So Ruby looks like a good candidate to replace Bash in build scripts. Maybe on Linux too.
It has bigger fan base than Perl, and… Homebrew uses Ruby - that might simplify copy-pasting of buildPhase from there
PS: also, embedding MRuby into Nix might allow syntax like
buildPhase = do
# ruby code here
end;
instead of concatenating opaque strings with Bash inside (“do” can also be read as “here begins something imperative” by people used to Haskell)
don’t you need admin access to create symlinks on Windows? this basically ruins any attempt at scripting symlinks, and why msys2 has a really bad workaround (it copies files instead).
IMHO adding an imperative DSL directly to nix makes the most sense, why can’t we just use nix directly instead of using an alternative language, embedded inside a nix-script?
Python is terrible for scripting external apps, I really hate using the subprocess module.
To make another bad suggestion: how about haskell-turtle? Turtle.Tutorial
yes, it needs elevated privileges, but we need to get them anyway to create C:\nix
And you only need it once, because the permissions w/r/t creating symlinks on Windows is configurable. You could have the Nix installer grant those permissions to the build users
Nobody answered your question.
why can’t we just use nix directly instead of using an alternative language, embedded inside a nix-script?
Nix is a pure expression language. It has no runtime that could cause arbitrary side-effects like invoking a compiler.
Some sort of light-weight functional language which does have one would be really cool to at least have as an option for use inside a derivation though IMO.
Echoing others; I want to use Nix professionally but a massive portion of my users won’t bite until it supports windows natively. WSL, and WSL2 are not installed on machines by default and require users to create a new mental model of their machine.
Nix is already a new “mental model” and in many aspects much bigger than WSL2 on windows. WSL2 seems to become pretty typical software to be installed. I have positive experience advertising and deploying NixOS for professional usage on WSL2 for employees on windows. Though, if it’s about selling products with Nix as a deployment tool then WSL2 is hardly an option and you’re much better off compile with Nix mingw cross-toolchain.
Yes, besides the compiler CI that is my main shortter term goal for Nix on Windows, I am interested in getting Nix working on regular non-developer Nix machines for e.g, dreams like allowing governments to migrate to free software more incrementally. So this “regular user, no time for WSL” use-case is I think is important and not to be dismissed.
Agreed but I can get someone to install a binary and execute one command easier than I can get them to install WSL, create a new linux user, pick a wsl shell, understand where in the heck they are in the filesystem, and then install nix and learn a whole other mental model.
Since I think shell languages make the most sense for calling external programs used for building software, I went through every active entry on the Oil shell project’s (fairly comprehensive!) list of alternative shells to see which of them run on Windows. One in particular stood out (already mentioned here by @symphorien) GSH, a non-interactive POSIX shell for Windows:
GSH is an implementation of a POSIX shell developed for the Windows platform. […]
GSH can be used to compile projects depending on autotools, UNIX make, [etc.] … [GSH] does not depend on the ‘fork system call’ and doesn’t emulate that system call (as it’s done on Cygwin).
It’s not complete yet:
As the project is still not complete you still need a Cygwin installation for the tools not provided by GSH. The only requirement is that the build should be done in a path for which Cygwin path maps directly to a Windows path.
But it still seems quite interesting, since it would be much more similar to the Bash builders we’re using on Unix-like platforms.
Another shell language with native Windows support that stood out as something potentially harmonious with the current Nixpkgs practice of using shell builders was Batsh, a programming language that compiles to GNU Bash and Windows BAT.
Other shell languages from the list that seem easy to build and deploy on Windows (i.e., no special environment like Cygwin or heavyweight language runtime is needed):
Other shell languages that advertise native (non-WSL) Windows support:
Again: the problem is exactly here - you will need to write (or to fix) almost all those external programs due to their lack on Windows.
A notable example is: you cannot code cp -r logic on shell, you need to call cp as an external program, and there is NO external program which can correctly do cp -r of a complex tree (such as LLVM or Chromium sources)
There is no ln -s which can handle paths longer than 255 characters, etc
So you think there are just going to be tons of little issues like this, so that using a shell-like language won’t make much sense at all?
Do you think it might be interesting or useful to see how annoying it is to write a cp -r that works on the Chromium sources on Windows in some of those shell languages, and maybe also Rakudo? That seems like something that might be fun for me to try once I have a little time.
I did in Perl5 (nix-windows discussed above was using Perl5 as stdenv.shell) and even that required a 3rd party module Win32::LongPath and patches. (Chef did something similar for Ruby: https://github.com/chef/chef/tree/main/lib/chef/win32). Rakudo’s only advantage is it has a bit better built-in support for Windows long paths and symlinks,
PowerShell Core (the open-source PowerShell implementation that gets all the cool new stuff, and has Linux and macOS ports) also always supports long file paths, even when the GPO setting on Windows for it is not enabled, and PowerShell includes built-ins for everything normally covered by coreutils.
It depends on .NET, so that sucks, and it may have other quirks. But the builtins supposedly will handle long paths, at least, and it’s probably better than depending on all of WSL.
I didn’t find any real documentation for this except this issue that notes that the old workaround (UNC paths starting with \\?\) is not needed on PowerShell Core as it is on the older distribution of PowerShell (‘Windows PowerShell’).
I recently stumbled upon duckscript | Simple, extendable and embeddable scripting language. and while it’s probably not an option worth considering due to its quiet verbose syntax, it should at least be relatively cross-platform and way less heavy than Rakudo.
nushell is getting good, fast. The folks working on it seems serious, passionate and experienced in a way that I trust it’s going places. It’s cross-platform, it’s Rust, I think there might even have some potentially interesting places where we could do much better than passing everything as strings between nix<->shell.
The latest release has a lot of interesting changes. I thought this video was a nice snapshot that really gave me a good, rare vibe that I’ve learned to trust a bit: https://www.youtube.com/watch?v=-lWL7Nc-OX4
I still can’t seem to catch up with everything, but I want to prototype replacing some of my deployment scripts and internal nix->shell with nushell and see how it feels.
Ruby - a powerhouse of a language that is easy to bootstrap