Is anyone trying to port nix and nixpkgs to output native Windows PE executables? I would like to avoid cross-compilation and cygwin if possible to get something as close as native as possible.
I talked a bit with @edolstra and he said the following things:
Two years ago I was using mainly linux → mingw cross. (and in some cases compilation inside wine ) but it seems you don’t want that, and in my case it was not really complicated dependency tree (a few dozen packages IIRC).
angerman (Moritz Angermann) · GitHub has linux → mingw cross more recently. I want to try clang cross at some point as it supports the MSVC++ ABI, and is generally more polished. (Major cross platform projects like chromium and firefox use clang for their windows builds now.)
Why avoid cross compilation? it seems like the best bet here. Otherwise you’d have to go through all of the issues of getting Nix runtime and bootstrapping to work on Windows.
It’s definitely not going to work for everything but we have some packages building for windows.
I would also like to have Nix running on Windows natively. Not for building, but for testing the Windows builds.
I looked at @aszlig’s code (linked above) and have to say it’s an amazing piece of work! Unfortunately it bitrotted due to both nixpkgs changes, Windows changes, and Cygwin changes.
The nixpkgs changes are pretty easy to fix. Mostly around cross compilation, vmTools, and make-iso9660-image.
Windows completely changed its unattended setup since this code was written. It’s possible to convert to the Autounattend.xml format and I have done most of that. The main problem I had is that there doesn’t appear to be a good way to skip the new “Meet Cortana” screen at first login.
My cross-compiled statically linked version of Cygwin Setup.exe did not work on Windows 10 (it just exits at startup). I tried building a new version of cygwin-setup but they have added a dependency on libsolv which is hard to build.
I think the main difficulty is that Microsoft regularly change and break the Windows unattended setup. There is no way of pinning to a specific build of the Windows ISO because it must be downloaded from the Microsoft website.
Cross-compilation is the local maxima. We have to look past the least effort route and introduce native Windows support. It’s going to pay off 100x in the future.
Business-wise, most people I talk to are interested in Nix. They understand the business needs and what it could bring to them in terms of guarantees and control. As soon as I mention that it doesn’t have Windows support the interest goes away. It’s why for example Anaconda has been built, there is a big pool of Windows developers that need native support for python packages. Anaconda borrowed Nix’s design and made it work for Windows.
As a (hypothetical) Windows developer I don’t want to install a Linux VM to build my Windows binaries. Or run Wine to test the cross-compiled binary in my Linux VM. It needs to integrate with Visual Studio and IntelliJ.
I know this doesn’t appeal to the Linux crowd. I myself don’t really care for Windows. But I like the idea of having Nix running on all the platforms.
Cross compile Nix to Windows. You can try it now in Nixpkgs with: nix-build -A pkgsCross.mingwW64.nix '<nixpkgs>'
Lots of missing dependencies though.
Natively compile Nix on Window using a native toolchain. Will probably need something like msys for compatibility with configure/make/autoconf. Otherwise you could write your own scripts (configure.bat) and compile with for instance MSVC.
If Nix can be compiled, then maybe we can get some basic stuff to run. But without POSIX threads or a Unix filesystem, you’re going to need a lot of patches. Cygwin has worked in the past and maybe something like WSL can help?
Fix Nix to compile on the target (or cross-compile for bootstrap):
Windows path format support
Case-insensitive filesystem
Windows file handling
[Future] Fix the C++ code to compile with MS’ C++ compiler
[Future] See if C:\\Nix\store can be moved to another drive
[Future] Use Windows sandboxing features (for --option sandbox true)
Bootstrap a new stdenv
Learn about PE executable and how they load *.dll files, see if it’s possible to put absolute references (so it doesn’t load them from arbitrary locations). This might require a PE-equivalent of patchelf.
Create a bootstrap tarball with clang, coreutils, tar, curl, …
Create the actual stdenv under pkgs/stdenv/
Patch nixpkgs
Add a lot of if conditions of packages for windows support
Looks like basic stuff like uids, kill signals, etc. Pthreads could be a big issue though. C++11 has native threading support. Maybe we can switch Nix to that?
If nix on WSL works, and we are able to drive WSL to spawn native windows binaries with PATHs and other env vars set correctly, then what’s the upside in maintaining a ‘native’ windows version? Sorry for the probably dumb question…
No I think your original post had it right: use WSL instead of Cygwin to get Nix running (which I think your company (?) did: https://www.tweag.io/posts/2017-11-10-nix-on-wsl.html); then get Visual C++ in Nixpkgs; then build native Windows beasts using that setup. My point is just that the whole cross compilation thread seems like a tangent to me - interesting but unrelated to the first question of what’s the shortest path to getting Windows-native builds from Nix. Right?
I don’t have access to a WSL-enabled box right now but I’ll get one and try it out, see if I can demo what I mean.
Right, I agree that WSL seems like the shortest path. I’m looking forward to see how the demo will work out.
One reason for porting Nix natively would the to improve the developer experience. At the moment I don’t think that Visual Studio or JetBrains IDE would be able to integrate with Nix directly. That being said, maybe there are ways to make it work with WSL as well.
) but it seems you don’t want that, and in my case it was not really complicated dependency tree (a few dozen packages IIRC).
so I’m not motivated to really move this forward, but I’m interested in the situation anyway.