System with NixOS: how to add another extra distribution

Help
4

Thanks @tokudan and @symphorien,

In principle, I followed the manual:

BIOS systems

You must set the option boot.loader.grub.device to specify on which disk the GRUB boot loader is to be installed. Without it, NixOS cannot boot.

UEFI systems

You must set the option boot.loader.systemd-boot.enable to true . nixos-generate-config should do this automatically for new configurations when booted in UEFI mode.

So I have in my configuration.nix that

boot.loader.systemd-boot.enable = true;

It seems that most of the solutions are related with configure grub rather than systemd-boot as said @tokudan (although I did not full understand the difference). I think that I will try the idea of about OSProber, but then I have to move to grub.

About containers, this was my first approach, I tried lxc/lxd container, and I was able to use over the terminal, but I was not able to share the X-system within the container. That is why I am trying the more drastic (and less elegant solution).

Thanks for your ideas!!

5

Just note that I was able to install another distribution, in this case Manjaro. The Debian installer, at least automatically, wanted to create an MBR and swap in the new hard disk partition. Manjaro was able to detect swap partition and create a grub.efi file. So I decided to just kept the systemd boot system, and I copied the file GRUB.efi created by Manjaro and manually add an entry to systemd. So now I see that other distribution together with the list of rolling releases of NixOS… only that went I select Manjaro it takes ages to boot as it load Grub. But it works, and it is simple to add entries to systemd to live easily with NixOS!

2 Likes
6

I’m just curious, what use case wasn’t fulfilled with NixOS?

EDIT: I’m assuming it didn’t have to be a debian specific use case as manjaro is in the arch family.

7

For what it’s worth, my grub and EFI config has always been

boot.loader.grub.enable = true;
boot.loader.grub.device = "nodev";
boot.loader.grub.efiSupport = true;
boot.loader.efi.canTouchEfiVariables = true;

and a vfat boot partition.

Only thing extra you’d need is boot.loader.grub.useOSProber.

8

If this is true, we should change wording on NixOS 23.11 manual | Nix & NixOS

it currently states:

UEFI systems
You must set the option boot.loader.systemd-boot.enable to true . nixos-generate-config should do this automatically for new configurations when booted in UEFI mode.

9

I end up avoiding OSProber and managing it manually because it both misses certain installations (I’ve never gotten it to detect Fedora) and adds 20 or so seconds to my nix rebuild.

10

Why not just use a container on your NixOS? “Systemd-nspawn” is your magic friend! :slight_smile:

Install debootstrap:

nix-env --install debootstrap

Create a directory in your $HOME:

mkdir ~/container/debian1 -p

Create the Debian environment:

sudo debootstrap --include=systemd-container --components=main,universe buster ~/container/debian1 https://deb.debian.org/debian

Edit “~/container/debian1/etc/securetty” and add:

pts/0
pts/1
pts/2
pts/3
pts/4
pts/5
pts/6
pts/7
pts/8
pts/9

Start the container:

sudo systemd-nspawn -b -D ~/container/debian1

Open a second terminal and change the root password:

sudo machinectl shell root@debian1 /bin/bash
passwd root
reboot

Now you can login on the console.

edit:
If you get an error, try this:

sudo machinectl stop debian1

sudo umount ~/container/debian/sys
sudo umount ~/container/debian/proc

https://wiki.archlinux.org/index.php/systemd-nspawn

3 Likes
11

Thanks for your new comments!

To overcome some issues, and to have a similar working environment that my colleges, I decided install other distribution in a new partition… As Ubuntu is the mandatory distribution in my job, Debian was a good choice. However, the installer gave some problems due to my motherboard (few components need not-free firmware with are not in the default DVD and need additional stuff, a bad choice of my hardware, not an issue of Debian that has a nice philosophy). So to solve the Julia language issue above, a friend was using Manjaro and at least at home I can use smoothly Julia.

My first approach was indeed containers and I tried lxc/lxd. It was successful to have a CLI environment, but I needed also a graphical one to visualize results of Julia calculations. I know that some people were able to have X-windows in containers, but most of the tips I found did not work directly in NixOS (and again I have to spend additional time that now I don’t have).

So, I am happy with NixOS, and it is my main distribution on the laptop and home-desktop but also it is true that you have to spend more energy for few tasks that are easier in others.

By the way, it is pretty easy to add a new entry to systemd manually and have other distribution that created a .efi file on your initial menu.

Thanks!

1 Like
12

The nix expression below gives you a drop in replacement for julia that allows you to use the package manager from within julia. Just add it to your overlays and then you can reference it from other nix expression. I haven’t found a package yet that doesn’t work with the expression below. Does this not work for you?

Edit: The expression assumes you want CUDA. Not sure if you do though.

{pkgs, stdenv, ...}:

with pkgs;
let
  julia = julia_11;
  d = version: "v${lib.concatStringsSep "." (lib.take 2 (lib.splitString "." version))}";
  extraLibs = [
    # IJulia.jl
    mbedtls
    zeromq3
    # ImageMagick.jl
    imagemagickBig
    # HDF5.jl
    hdf5
    # Cairo.jl
    cairo
    gettext
    pango.out
    glib.out
    # Gtk.jl
    gtk3
    gdk_pixbuf
    # GZip.jl # Required by DataFrames.jl
    gzip
    zlib
    # GR.jl # Runs even without Xrender and Xext, but cannot save files, so those are required
    xorg.libXt
    xorg.libX11
    xorg.libXrender
    xorg.libXext
    glfw
    freetype
    # Flux.jl
    cudatoolkit
    linuxPackages.nvidia_x11
    git gitRepo gnupg autoconf curl
    procps gnumake utillinux m4 gperf unzip
    libGLU_combined
    xorg.libXi xorg.libXmu freeglut
    xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib
    ncurses5 stdenv.cc binutils
    # Arpack.jl
    arpack
    gfortran.cc
    (pkgs.runCommand "openblas64_" {} ''
      mkdir -p "$out"/lib/
      ln -s ${openblasCompat}/lib/libopenblas.so "$out"/lib/libopenblas64_.so.0
    '')
  ];
in
stdenv.mkDerivation rec {
  name = "julia-env";
  version = julia.version;
  nativeBuildInputs = [ makeWrapper cacert git pkgconfig which ];
  buildInputs = [
    julia
    /* jupyterEnv  # my custom jupyter */
  ] ++ extraLibs;
  phases = [ "installPhase" ];
  installPhase = ''
    export CUDA_PATH="${cudatoolkit}"
    export LD_LIBRARY_PATH=${lib.makeLibraryPath extraLibs}
    # pushd $JULIA_PKGDIR/${d version}
    makeWrapper ${julia}/bin/julia $out/bin/julia \
        --prefix LD_LIBRARY_PATH : "$LD_LIBRARY_PATH" \
        --prefix LD_LIBRARY_PATH ":" "${linuxPackages.nvidia_x11}/lib" \
        --set CUDA_PATH "${cudatoolkit}" \
        --set JULIA_PKGDIR $JULIA_PKGDIR
        # --set JULIA_LOAD_PATH $JULIA_PKGDIR/${d version}
  '';
}
2 Likes
13

Thanks! I have tried some similar and worked but only partially. I am not familiar about how to add those overlays correctly on my configuration.nix, and I wasn’t able to diagnostic/ascertain why it did not work. But I will try again.

14

Assuming you are using ~/.config/nixpkgs/overlays.nix for your overlays than something like this

[
(self: super: {
julia = super.callPackage /path/to/the/nix/expression/above/julia-env.nix { };
})
]

should work.

1 Like
15

Thanks,

I hava the same problem that I had before: installing dependences related with Python. Julia tries to create a kind of python environment using miniconda, that is created by Conda.jl
This step is always failing for me, :frowning_face:

16

Like are you trying to use a Julia package that depends on PyCall like PyPlot?
PyCall is fairly finicky to setup, but the expression below gives you a julia environment that works with PyPlot

with import {};

let

myPackages = pythonPackages: with pythonPackages; [
matplotlib # PyPlot needs a Python with matplotlib
];
python-stuff = python3.withPackages myPackages;

in

pkgs.stdenv.mkDerivation {
name = “julia-env”;
buildInputs = with pkgs; [
julia # This uses the Julia we have just defined in the overlay before
python-stuff
];
shellHook = ‘’
# Set PYTHONPATH so that PyCall, PyPlot etc. in julia finds the relevant packages
export PYTHONPATH=${python-stuff}/lib/python3.7/site-packages/
# Configure PyCall to pick up the correct python binary
julia -e ‘ENV[“PYTHON”]=“${python-stuff}/bin/python”; using Pkg; Pkg.activate(“./”); Pkg.build(“PyCall”)’
‘’;
}

The ENV[“PYTHON”] part is possibly unnecessary (after you have compiled PyCall?) if you already have the correct Python path setup but I couldn’t be bothered testing it properly.

17

There are several packages that try to install python packages: pyplot, NetCDFs, NCDatasets,…

In general the error began with

  Building Conda ─→ `~/.julia/packages/Conda/3rPhK/deps/build.log`
  Building PyCall → `~/.julia/packages/PyCall/kAhnQ/deps/build.log`
┌ Error: Error building `PyCall`: 
│   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
│                                  Dload  Upload   Total   Spent    Left  Speed
100 68.4M  100 68.4M    0     0  37.6M      0  0:00:01  0:00:01 --:--:-- 37.5M
│ PREFIX=/home/aang/.julia/conda/3
│ Unpacking payload ...
│ /home/aang/.julia/conda/3/installer.sh: line 351: /home/aang/.julia/conda/3/conda.exe: No such file or directory
│ /home/aang/.julia/conda/3/installer.sh: line 353: /home/aang/.julia/conda/3/conda.exe: No such file or directory
│ ┌ Info: No system-wide Python was found; got the following error:
18

So I have chased down the source of the error. Essentially it comes down to the package BinDeps being incompatible with a standard julia setup on Nixos (add support for the Nix package manager by rened · Pull Request #115 · JuliaPackaging/BinDeps.jl · GitHub).
However, tbenst seems to have posted a working julia env for this: use with `nix-shell julia-shell.nix` · GitHub

On the upside, BinDeps is also being phased out and you could probably ask the package maintainers to migrate to the newer solution (BinaryProvider) that works on nix.

1 Like
19

Hi @cstich,

I have just commented your solution on the Julia Language Discourse.

Build Julia on NixOS - General Usage - Julia Programming Language

Thanks.

20

So I tried using the nix expressions above, and it doesn’t seem like GR is actually working with it.

julia> Pkg.add("GR")
   Cloning default registries into `~/.julia`
   Cloning registry from "https://github.com/JuliaRegistries/General.git"
     Added registry `General` to `~/.julia/registries/General`
 Resolving package versions...
 Installed GR ─ v0.47.0
  Updating `~/.julia/environments/v1.1/Project.toml`
  [28b8d3ca] + GR v0.47.0
  Updating `~/.julia/environments/v1.1/Manifest.toml`
  [28b8d3ca] + GR v0.47.0
  [2a0f44e3] + Base64 
  [8bb1440f] + DelimitedFiles 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [a63ad114] + Mmap 
  [de0858da] + Printf 
  [9a3f8284] + Random 
  [ea8e919c] + SHA 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [8dfed614] + Test 
  [cf7118a7] + UUIDs 
  [4ec0a83e] + Unicode 
  Building GR → `~/.julia/packages/GR/tPkHV/deps/build.log`

julia> using GR
[ Info: Precompiling GR [28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71]

julia> histogram(randn(10000))
sh: /home/sean/.julia/packages/GR/tPkHV/src/../deps/gr/bin/gksqt: No such file or directory
connect: Connection refused
GKS: can't connect to GKS socket application
Did you start 'gksqt'?

GKS: Open failed in routine OPEN_WS
GKS: GKS not in proper state. GKS must be either in the state WSOP or WSAC in routine ACTIVATE_WS
GKS: GKS not in proper state. GKS must be either in the state WSAC or SGOP in routine POLYLINE
... # It continues like this for a bit...

julia> exit()

~/projects/julia_test 
❯ ldd ~/.julia/packages/GR/tPkHV/deps/gr/bin/gksqt
        linux-vdso.so.1 (0x00007ffdf2fb6000)
        libQtGui.so.4 => not found
        libQtNetwork.so.4 => not found
        libQtCore.so.4 => not found
        libpthread.so.0 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libpthread.so.0 (0x00007fc4d3fc9000)
        libstdc++.so.6 => not found
        libm.so.6 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libm.so.6 (0x00007fc4d3e33000)
        libgcc_s.so.1 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libgcc_s.so.1 (0x00007fc4d3c1b000)
        libc.so.6 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libc.so.6 (0x00007fc4d3a65000)
        /lib64/ld-linux-x86-64.so.2 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib64/ld-linux-x86-64.so.2 (0x00007fc4d3fec000)

~/projects/julia_test
❯ cat shell.nix 
{ pkgs ? import <nixpkgs> {} }:

with pkgs;

let
  # ./julia.nix contains the posted nix expression in this thread, "julia-env".
  my-julia = callPackage ./julia.nix {};
in

stdenv.mkDerivation {
  name = "${my-julia.version}-env";
  buildInputs = [
    my-julia
  ];
}

Near as I can tell, GR is building an additional binary that doesn’t have the requisite libraries linked in? Still kind of figuring out Nix as I go though, so it’s possible I’m just missing something.

21

@sean-bennett112 Looks like GR depends on a few Qt libraries, which you will need to add to the extraLibs list.

You should be able to do nix-locate -1 -w foo.so.4 to find which nix package provides the shared library. For a bit more context see the relevant section in the Nixos wiki.

Edit: Grammar

22

@cstich Thanks for the nix-locate pointer, that’s incredibly useful!

After doing that, looks like things still don’t work. After some poking around, it looks like this is due to the interpreter being incorrect.

❯ readelf -l ~/.julia/packages/GR/tPkHV/deps/gr/bin/gksqt | grep Requesting
      [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]

As you can see, it’s expecting an interpreter at /lib64/ld-linux-x86-64.so.2 which does not exist on NixOS. I’ve confirmed that manually patching the binary(after running Pkg.add("GR"))via the below command does resolve the issue.

patchelf --set-interpreter "(cat $NIX_CC/nix-support/dynamic-linker)" ~/.julia/packages/GR/tPkHV/deps/gr/bin/gksqt

For GR specifically, you can see in build.jl that we can pass in an environment variable GRDIR to use a pre-existing copy of gksqt. From a cursory glance through nixpkgs, it looks like GR framework isn’t packaged yet. That’s a solvable problem, though.

However I’m not sure if there exists a more general solution for when Julia packages download pre-built binaries. Maybe this is something the move towards BinaryProvider will help address?

In any case, thanks for pointing me in the right direction!

23

If someone else eventually needs to patch the gksqt binary. Here is how you do it semi-automatically within a nix expression:

let 
 libPath = lib.makeLibraryPath [
    qt4 
    gcc9 
    stdenv.cc.cc.lib
  ];

in

  pkgs.stdenv.mkDerivation {
  name = "sandbox-julia";
  buildInputs = with pkgs; [
    julia
  ];
  shellHook = ''
    # Make sure the GR package is installed in the current project
    julia -e 'using Pkg; Pkg.activate("./"); Pkg.add("GR")'

  # Patch the GKS binary for GR
     patchelf \
     --set-interpreter ${glibc}/lib/ld-linux-x86-64.so.2 \
     --set-rpath "${libPath}" \
     /home/user/.julia/packages/GR/cRdXQ/deps/gr/bin/gksqt
'';
}
2 Likes