System with NixOS: how to add another extra distribution

Hey,

Currently, I have NixOS as my main Linux distribution (on my laptop and desktop). However, as in my job I have to use Ubuntu/Debian, I will need to have also installed at home.

So I am trying to install this extra distribution (first I tried Debian) on a free partition of one internal hard-drive. I was able to do, but the problem is that there is no entry in my systemd-boot generated by NixOS with the Debian distribution. I could boot Debian, but only by setting it temporally on the firmware of the computer. So essentially when I installed Debian then the installer created in the free partition three others: ‘/’, swap and another small FAT partition for grub (I guess) which is the one I have to manually set on the firmware if I would like to run it.

Does anyone have experience installing an extra distribution on a NixOS system, and add it to the systemd-boot? I have found a lot of information about how to do the opposite: install NixOS with the pre-existent Ubuntu/Debian (but not Ubuntu/Debian with a pre-existent NixOS).

Thanks!

Not my own experience, but it appears that the grub module supports that. Check NixOS Search - Loading...
I guess there are some pitfalls, e.g. having Debian/Ubuntu overwrite the NixOS boot leader, so I hope others chime in.
I guess it should be possible with systemd-boot as well, but I don’t see that in the module.

As an afterthought: Maybe it makes sense to run those distributions in a container in NixOS? I’ve done that on a server. Depends on how strict you want the split to be.

1 Like

You could also use grub with os-prober:
https://nixos.org/nixos/options.html#boot.loader.grub.useosprober

This allows you not to have to find the right syntax for grub entries.

2 Likes

Thanks @tokudan and @symphorien,

In principle, I followed the manual:

BIOS systems

You must set the option boot.loader.grub.device to specify on which disk the GRUB boot loader is to be installed. Without it, NixOS cannot boot.

UEFI systems

You must set the option boot.loader.systemd-boot.enable to true . nixos-generate-config should do this automatically for new configurations when booted in UEFI mode.

So I have in my configuration.nix that

boot.loader.systemd-boot.enable = true;

It seems that most of the solutions are related with configure grub rather than systemd-boot as said @tokudan (although I did not full understand the difference). I think that I will try the idea of about OSProber, but then I have to move to grub.

About containers, this was my first approach, I tried lxc/lxd container, and I was able to use over the terminal, but I was not able to share the X-system within the container. That is why I am trying the more drastic (and less elegant solution).

Thanks for your ideas!!

Just note that I was able to install another distribution, in this case Manjaro. The Debian installer, at least automatically, wanted to create an MBR and swap in the new hard disk partition. Manjaro was able to detect swap partition and create a grub.efi file. So I decided to just kept the systemd boot system, and I copied the file GRUB.efi created by Manjaro and manually add an entry to systemd. So now I see that other distribution together with the list of rolling releases of NixOS… only that went I select Manjaro it takes ages to boot as it load Grub. But it works, and it is simple to add entries to systemd to live easily with NixOS!

2 Likes

I’m just curious, what use case wasn’t fulfilled with NixOS?

EDIT: I’m assuming it didn’t have to be a debian specific use case as manjaro is in the arch family.

For what it’s worth, my grub and EFI config has always been

boot.loader.grub.enable = true;
boot.loader.grub.device = "nodev";
boot.loader.grub.efiSupport = true;
boot.loader.efi.canTouchEfiVariables = true;

and a vfat boot partition.

Only thing extra you’d need is boot.loader.grub.useOSProber.

If this is true, we should change wording on NixOS - NixOS 21.05 manual

it currently states:

UEFI systems
You must set the option boot.loader.systemd-boot.enable to true . nixos-generate-config should do this automatically for new configurations when booted in UEFI mode.

I end up avoiding OSProber and managing it manually because it both misses certain installations (I’ve never gotten it to detect Fedora) and adds 20 or so seconds to my nix rebuild.

Why not just use a container on your NixOS? “Systemd-nspawn” is your magic friend! :slight_smile:

Install debootstrap:

nix-env --install debootstrap

Create a directory in your $HOME:

mkdir ~/container/debian1 -p

Create the Debian environment:

sudo debootstrap --include=systemd-container --components=main,universe buster ~/container/debian1 https://deb.debian.org/debian

Edit “~/container/debian1/etc/securetty” and add:

pts/0
pts/1
pts/2
pts/3
pts/4
pts/5
pts/6
pts/7
pts/8
pts/9

Start the container:

sudo systemd-nspawn -b -D ~/container/debian1

Open a second terminal and change the root password:

sudo machinectl shell root@debian1 /bin/bash
passwd root
reboot

Now you can login on the console.

edit:
If you get an error, try this:

sudo machinectl stop debian1
sudo umount ~/container/debian/sys
sudo umount ~/container/debian/proc

https://wiki.archlinux.org/index.php/systemd-nspawn

3 Likes

Thanks for your new comments!

  • In principle, I avoided the grub because the information on the manual about UEFI systems.

  • NixOS is fine, but most of the things, but sometimes is needs additional work for things that are simple on others distributions:

To overcome some issues, and to have a similar working environment that my colleges, I decided install other distribution in a new partition… As Ubuntu is the mandatory distribution in my job, Debian was a good choice. However, the installer gave some problems due to my motherboard (few components need not-free firmware with are not in the default DVD and need additional stuff, a bad choice of my hardware, not an issue of Debian that has a nice philosophy). So to solve the Julia language issue above, a friend was using Manjaro and at least at home I can use smoothly Julia.

My first approach was indeed containers and I tried lxc/lxd. It was successful to have a CLI environment, but I needed also a graphical one to visualize results of Julia calculations. I know that some people were able to have X-windows in containers, but most of the tips I found did not work directly in NixOS (and again I have to spend additional time that now I don’t have).

So, I am happy with NixOS, and it is my main distribution on the laptop and home-desktop but also it is true that you have to spend more energy for few tasks that are easier in others.

By the way, it is pretty easy to add a new entry to systemd manually and have other distribution that created a .efi file on your initial menu.

Thanks!

1 Like

The nix expression below gives you a drop in replacement for julia that allows you to use the package manager from within julia. Just add it to your overlays and then you can reference it from other nix expression. I haven’t found a package yet that doesn’t work with the expression below. Does this not work for you?

Edit: The expression assumes you want CUDA. Not sure if you do though.

{pkgs, stdenv, ...}:

with pkgs;
let
  julia = julia_11;
  d = version: "v${lib.concatStringsSep "." (lib.take 2 (lib.splitString "." version))}";
  extraLibs = [
    # IJulia.jl
    mbedtls
    zeromq3
    # ImageMagick.jl
    imagemagickBig
    # HDF5.jl
    hdf5
    # Cairo.jl
    cairo
    gettext
    pango.out
    glib.out
    # Gtk.jl
    gtk3
    gdk_pixbuf
    # GZip.jl # Required by DataFrames.jl
    gzip
    zlib
    # GR.jl # Runs even without Xrender and Xext, but cannot save files, so those are required
    xorg.libXt
    xorg.libX11
    xorg.libXrender
    xorg.libXext
    glfw
    freetype
    # Flux.jl
    cudatoolkit
    linuxPackages.nvidia_x11
    git gitRepo gnupg autoconf curl
    procps gnumake utillinux m4 gperf unzip
    libGLU_combined
    xorg.libXi xorg.libXmu freeglut
    xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib
    ncurses5 stdenv.cc binutils
    # Arpack.jl
    arpack
    gfortran.cc
    (pkgs.runCommand "openblas64_" {} ''
      mkdir -p "$out"/lib/
      ln -s ${openblasCompat}/lib/libopenblas.so "$out"/lib/libopenblas64_.so.0
    '')
  ];
in
stdenv.mkDerivation rec {
  name = "julia-env";
  version = julia.version;
  nativeBuildInputs = [ makeWrapper cacert git pkgconfig which ];
  buildInputs = [
    julia
    /* jupyterEnv  # my custom jupyter */
  ] ++ extraLibs;
  phases = [ "installPhase" ];
  installPhase = ''
    export CUDA_PATH="${cudatoolkit}"
    export LD_LIBRARY_PATH=${lib.makeLibraryPath extraLibs}
    # pushd $JULIA_PKGDIR/${d version}
    makeWrapper ${julia}/bin/julia $out/bin/julia \
        --prefix LD_LIBRARY_PATH : "$LD_LIBRARY_PATH" \
        --prefix LD_LIBRARY_PATH ":" "${linuxPackages.nvidia_x11}/lib" \
        --set CUDA_PATH "${cudatoolkit}" \
        --set JULIA_PKGDIR $JULIA_PKGDIR
        # --set JULIA_LOAD_PATH $JULIA_PKGDIR/${d version}
  '';
}
2 Likes

Thanks! I have tried some similar and worked but only partially. I am not familiar about how to add those overlays correctly on my configuration.nix, and I wasn’t able to diagnostic/ascertain why it did not work. But I will try again.

Assuming you are using ~/.config/nixpkgs/overlays.nix for your overlays than something like this

[
(self: super: {
julia = super.callPackage /path/to/the/nix/expression/above/julia-env.nix { };
})
]

should work.

1 Like

Thanks,

I hava the same problem that I had before: installing dependences related with Python. Julia tries to create a kind of python environment using miniconda, that is created by Conda.jl
This step is always failing for me, :frowning_face:

Like are you trying to use a Julia package that depends on PyCall like PyPlot?
PyCall is fairly finicky to setup, but the expression below gives you a julia environment that works with PyPlot

with import {};

let

myPackages = pythonPackages: with pythonPackages; [
matplotlib # PyPlot needs a Python with matplotlib
];
python-stuff = python3.withPackages myPackages;

in

pkgs.stdenv.mkDerivation {
name = “julia-env”;
buildInputs = with pkgs; [
julia # This uses the Julia we have just defined in the overlay before
python-stuff
];
shellHook = ‘’
# Set PYTHONPATH so that PyCall, PyPlot etc. in julia finds the relevant packages
export PYTHONPATH=${python-stuff}/lib/python3.7/site-packages/
# Configure PyCall to pick up the correct python binary
julia -e ‘ENV[“PYTHON”]="${python-stuff}/bin/python"; using Pkg; Pkg.activate("./"); Pkg.build(“PyCall”)’
‘’;
}

The ENV[“PYTHON”] part is possibly unnecessary (after you have compiled PyCall?) if you already have the correct Python path setup but I couldn’t be bothered testing it properly.

There are several packages that try to install python packages: pyplot, NetCDFs, NCDatasets,…

In general the error began with

  Building Conda ─→ `~/.julia/packages/Conda/3rPhK/deps/build.log`
  Building PyCall → `~/.julia/packages/PyCall/kAhnQ/deps/build.log`
┌ Error: Error building `PyCall`: 
│   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
│                                  Dload  Upload   Total   Spent    Left  Speed
100 68.4M  100 68.4M    0     0  37.6M      0  0:00:01  0:00:01 --:--:-- 37.5M
│ PREFIX=/home/aang/.julia/conda/3
│ Unpacking payload ...
│ /home/aang/.julia/conda/3/installer.sh: line 351: /home/aang/.julia/conda/3/conda.exe: No such file or directory
│ /home/aang/.julia/conda/3/installer.sh: line 353: /home/aang/.julia/conda/3/conda.exe: No such file or directory
│ ┌ Info: No system-wide Python was found; got the following error:

So I have chased down the source of the error. Essentially it comes down to the package BinDeps being incompatible with a standard julia setup on Nixos (add support for the Nix package manager by rened · Pull Request #115 · JuliaPackaging/BinDeps.jl · GitHub).
However, tbenst seems to have posted a working julia env for this: use with `nix-shell julia-shell.nix` · GitHub

On the upside, BinDeps is also being phased out and you could probably ask the package maintainers to migrate to the newer solution (BinaryProvider) that works on nix.

1 Like

Hi @cstich,

I have just commented your solution on the Julia Language Discourse.

Build Julia on NixOS - General Usage - JuliaLang

Thanks.

So I tried using the nix expressions above, and it doesn’t seem like GR is actually working with it.

julia> Pkg.add("GR")
   Cloning default registries into `~/.julia`
   Cloning registry from "https://github.com/JuliaRegistries/General.git"
     Added registry `General` to `~/.julia/registries/General`
 Resolving package versions...
 Installed GR ─ v0.47.0
  Updating `~/.julia/environments/v1.1/Project.toml`
  [28b8d3ca] + GR v0.47.0
  Updating `~/.julia/environments/v1.1/Manifest.toml`
  [28b8d3ca] + GR v0.47.0
  [2a0f44e3] + Base64 
  [8bb1440f] + DelimitedFiles 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [a63ad114] + Mmap 
  [de0858da] + Printf 
  [9a3f8284] + Random 
  [ea8e919c] + SHA 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [8dfed614] + Test 
  [cf7118a7] + UUIDs 
  [4ec0a83e] + Unicode 
  Building GR → `~/.julia/packages/GR/tPkHV/deps/build.log`

julia> using GR
[ Info: Precompiling GR [28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71]

julia> histogram(randn(10000))
sh: /home/sean/.julia/packages/GR/tPkHV/src/../deps/gr/bin/gksqt: No such file or directory
connect: Connection refused
GKS: can't connect to GKS socket application
Did you start 'gksqt'?

GKS: Open failed in routine OPEN_WS
GKS: GKS not in proper state. GKS must be either in the state WSOP or WSAC in routine ACTIVATE_WS
GKS: GKS not in proper state. GKS must be either in the state WSAC or SGOP in routine POLYLINE
... # It continues like this for a bit...

julia> exit()

~/projects/julia_test 
❯ ldd ~/.julia/packages/GR/tPkHV/deps/gr/bin/gksqt
        linux-vdso.so.1 (0x00007ffdf2fb6000)
        libQtGui.so.4 => not found
        libQtNetwork.so.4 => not found
        libQtCore.so.4 => not found
        libpthread.so.0 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libpthread.so.0 (0x00007fc4d3fc9000)
        libstdc++.so.6 => not found
        libm.so.6 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libm.so.6 (0x00007fc4d3e33000)
        libgcc_s.so.1 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libgcc_s.so.1 (0x00007fc4d3c1b000)
        libc.so.6 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libc.so.6 (0x00007fc4d3a65000)
        /lib64/ld-linux-x86-64.so.2 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib64/ld-linux-x86-64.so.2 (0x00007fc4d3fec000)

~/projects/julia_test
❯ cat shell.nix 
{ pkgs ? import <nixpkgs> {} }:

with pkgs;

let
  # ./julia.nix contains the posted nix expression in this thread, "julia-env".
  my-julia = callPackage ./julia.nix {};
in

stdenv.mkDerivation {
  name = "${my-julia.version}-env";
  buildInputs = [
    my-julia
  ];
}

Near as I can tell, GR is building an additional binary that doesn’t have the requisite libraries linked in? Still kind of figuring out Nix as I go though, so it’s possible I’m just missing something.