My team recently begun using Nix/NixOS as our main packager, OS and mechanism for deployment for our Elixir projects. We have a growing code base in BEAM languages, and we would like to use Nix more.
Making it work in Nix has been a journey, and now that we got there, we would like to help streamline that a bit upstream. We are ready to spent time and energy doing so. We have some ideas, but we would like that to be a community effort. There have been interest in Nix in the BEAM ecosystem, but the current state of the BEAM tooling in nix is not particularly helping convinced people through.
I am going to make this a 2 parts thing. First around the state of the ecosystem, then around the ideas we have. This is a WIP after some discussion with people that were active in the BEAM part of nixpkgs. I fully expect this to be a limited and partial view of the state of the ecosystem and to learn a lot from the community. It is meant to be a place to coordinate and organize before working on significant change.
If you see anything you disagree with or that seems factually incorrect… That is why i make this post ! Please tell us about it
State of the BEAM Ecosystem
pkgs/top-level/beam-packages.nixprovide some named version of erlang under
beam.interpreters. Among others, we find version with odbc, with the javac flags and some
_noxversion that are headless version without the graphic environement brought through the
:observertool. There is also the (probably now unmaintained) Basho fork of R16 for Riak. Some of these versions are also combined to offer elixir and LFE interpreters.
pkgs/top-level/beam-packages.nixalso provides a
packagesWithhelper to generate a package set with a specific version of erlang, see below for the meaning of the package set.
pkgs/development/beam-modulesprovides a ton of things. Deep down, it is meant to act as an interface with the BEAM community tooling, but i do not think it is well maintained or used by the community at this time.
- Mix and Erlang.mk “works as expected”. This means that when used as expected by the BEAM community, they will not work in sandboxed mode. They also have a “bootstrap” step that need to be run before using them that is not well documented in what it does or why. (which is related to
- BEAM packages mostly come from Hex. This is the package registry the BEAM community has converged on. There is a semi automatic tool to run them and copy builders for them into the nixpkgs repository in a snapshot. This seems to not be maintained, nor heavily used.
- Rebar3 is offered in two version. One of them is hermetic
rebar3and the other is the normal Rebar3 called
rebar3-open. This seems anterior to the widespread use of sandboxing.
- There is a
fetchHexfunction that allows for direct use of Hex package name and SHA256. That should work with sandboxing.
fetch-rebar-depsdoes not work outside of sandboxing.
- Deep down, the way to use Nix tooling seems to be that we need to handwrite all the dependencies in the nix derivation. This is kinda opposite to the current situation of both rebar3 and Mix, which use lockfile of their own, with proper hash, to define the dependencies.
- @Ankhers has been working on a tool that can parse a rebar lockfile and translate it into a nix file that could be imported.
Here are some propositions i think may make sense. The first two i think are better done sooner than later. They have high maintenance burden that are not done right now for very few benefit and they lock us for exploring better solutions.
- Get rid of the hermetic rebar3. Rename
rebar3". We have sandboxing by default now anyway.
- Get rid of the whole snapshot package set. This is not well maintained anyway. And
fetchHexcan use Hex directly.
- Explore simplifying the way we build elixir and LFE now that we do not have to maintain the package set
- Explore cleaning up the bootstrapping part for mix. If necessary work with the mix team to simplify it.
- Update the version of Hex in nixpkgs.
- Bring @Ankhers tool (
rebar32nix) in and provide an equivalent
mix2nix. Explore ways to build an intermediate derivation with it, or just import a
dep.nixgenerated with it.
- Update the documentation accordingly
- Explore the possibility to support the
scopefeature of nixpkgs.
- [New] Fix the doubling of
erl/libsin the path, producing a lot of double load warning in Elixir.