Hello,
I’m an enthusiastic Nix user whose day job is all in the Elixir ecosystem. I’d love to have the option to leverage Nix to package up our releases, especially to be able to capitalize on pkgs.dockerTools and other knock-on effects. We currently use asdf at development time and Docker at deploy-time and have to re-solve the same compatibility concerns in both layers.
I’m aware of these projects and find in practice that each of them is only part of the story:
-
hex2nix - last commit Aug '19, haven’t tried recently
-
mix2nix - last commit Apr '19, haven’t tried recently
-
mixnix - last commit Mar '19, incompatible with the current (undocumented)
mix.lock schema
-
Nixpkgs documentation including
buildHex and buildMix which don’t seem to get on all that well with a full-sized application, such as a Phoenix project
Is anyone familiar with ways to accentuate them to get to a happy answer today? Does anyone have a new project in this space that I can contribute to? I’m envious of the higher-level tooling available for Haskell users, Python users, even Node users, and especially of options like naersk and crate2nix for Rust applications.
I’d really like to be able to cite an example of taking a brand new mix phx.new project and spit out a release at the end with all of the downloads and compilation being done via nix-build instead.
In my spare time I’m plinking away at a standalone repo that I’ll probably turn into a flake/NUR/Cachix etc. that would provide the same catalog of historical versions that Elixir devs commonly get from asdf today, but it’s hard for me to keep up my motivation there when I get to “okay I have the exact version of Erlang and Elixir I wanted, now what?”
2 Likes
Sadly. This does seem to be a bit of a quagmire at present, with lots of the information out there being obsoleted.
We are heavy users of nix for our development environments, and generally interested in packaging Erlang applications with nix too. We do that today in one project by using rebar3 to build outside of nix, and then we have a nix derivation that uses the built asset as its source. That’s very much an expedient hack though, and not where we want to be.
I’d originally looked at using Announcing Rebar32nix, which, for Erlang, looks to me like the right approach, but wasn’t viable when I used it because most of the beam capabilities had already been removed from nixpkgs by that point.
There are also some prior discussions around this that are worth looking at, e.g. State of the BEAM Ecosystem in Nix - #6 by DianaOlympos
I’d be pretty interested in getting this conversation going again, and chiming in from the Erlang point of view. I don’t have any experience of Elixir, so I can’t comment about specifics there.
On another note, I maintain GitHub - id3as/nixpkgs-nixerl: A nix overlay for Erlang Releases, which I created as a sort-of replacement for kerl (which serves a similar function to asdf) in a nix world. It specifically makes most released versions of Erlang available for use.
1 Like
Hey @shanesveller happy to see you around 
So i have been in that state for quite a long time. Right now there are no easy solutions and all the X2nix afaik depends on reading the mix.lock which is… problematic as you pointed out.
The nix ecosystem in general has two ways to handle that.
- Reproduce the package manager into nix itself. This is what is happening for haskell or python as an example.
- Do a two stage derivation, one of them fetching the whole set of dependencies using the language package manager and the lock file to ensure that you get something reproducible out of it, then validating that with a hash to satisfy the sandbox. That is how
buildGoModule work as an example.
The previous solution for BEAM in nix, that i tore down last year, is to use option 1. It simply did not work out because it means regularly mirroring all of hex into nixpkgs. This need a maintenance level that is really hard to produce. As a big user of the Python ecosystem in nix, it is probably the source of most of my pain because even the Python ecosystem cannot cope.
So my current goal would be to provide a solution based on option 2. It is not particularly hard to build imho, in pure nix, it is just that i had no time to do it yet. Following the path of buildGoModule and copying their code a lot should be enough to get us going. It will need documentation and imho should be a direct PR to nixpkgs. That should be definitely doable for rebar3 too @philipstears.
This approach has problems too, it seems to have generated quite some frictions with some part of the nix community. But the other 2 options, ie depending on reading a lock file that is opaque, reverse engineering it or reproducing hex on nixpkgs, both seems like too heavy in term of maintenance to be viable.
I am ofc really open to other ideas 
I would particularly like if we could get this going because it would probably solve a lot of my problems with my own CI. I have just been a bit busy lately.
I would be happy to provide review and help if someone want to get on that ship though.
2 Likes
Hey @shanesveller, I don’t know if you’re on the Erlang slack (the link is at Community - Erlang/OTP if not), but a few folks (including @DianaOlympos and myself) have been talking about this on the “nix” channel there.
3 Likes
I originally wrote rebar32nix, but never really received any sort of feedback on it. So I just sort of let it go to waste. If there are people that are currently working with it, or want to, I am more than happy to continue work on it. I will also join the nix channel on the Erlang slack and maybe we can come up with something that works for everyone.
1 Like
Hey all, I have currently the same opinion as @DianaOlympos.
Reproducing the complete behavior of mix & rebar as a package manager within Nix seems a bit of too much work from my point of view (the .lock file is fine).
Even if I see the benefits this approach would bring - super super fast rebuilds in CI.
But in my work habit the dependence list in the .lock file is still countable and changes to my .lock file aren’t happening daily.
So a rebuild of + half a minute more is total acceptable occasionally.
But I have to explain my main usage or Nix with Elixir is the setup process for having a reproducible development environment.
So I’m aware in other main usage scenarios different things could be more valuable.
@shanesveller if you haven’t found my sample repo for showing off how I’m using Nix with Elixir currently → github.com/cw789/elixir_nix_seed
My future “goal” (not active working on it) for Nix & Elixir is cross compilation.
To have an easy setup where I can build releases and deploy on different architectures already including ERTS.
1 Like
As mix deps.get should provide (almost) constant output I have created the derivation that fetches all dependencies via mix dips.get:
{ stdenvNoCC, elixir, rebar, rebar3, git, cacert }:
let
fetchMixDeps =
{ name ? null, src, sha256, env ? "prod" }:
stdenvNoCC.mkDerivation {
name = "mix-deps" + (if name != null then "-${name}" else "");
nativeBuildInputs = [ elixir git cacert ];
inherit src;
MIX_ENV = env;
MIX_REBAR = "${rebar}/bin/rebar";
MIX_REBAR3 = "${rebar3}/bin/rebar3";
configurePhase = ''
export HEX_HOME="$PWD/hex"
export MIX_HOME="$PWD/mix"
export MIX_DEPS_PATH="$out"
export REBAR_GLOBAL_CONFIG_DIR="$PWD/rebar3"
export REBAR_CACHE_DIR="$PWD/rebar3.cache"
mix local.hex --force
'';
buildPhase = ''
mix deps.get
find "$out" -path '*/.git/*' -a ! -name HEAD -exec rm -rf {} +
'';
dontInstall = true;
outputHashAlgo = "sha256";
outputHashMode = "recursive";
outputHash = sha256;
impureEnvVars = stdenvNoCC.lib.fetchers.proxyImpureEnvVars;
};
in fetchMixDeps
Which is enough for my uses. It also do not rely on manually traversing mix.lock (which format is not guaranteed by the Mix in any way, so traversing it will always be “here be dragons”).
I use it for building ElixirLS in a form of:
{ stdenv, elixir, rebar3, hex, fetchFromGitHub, fetchMixDeps, git, cacert }:
let
json = builtins.fromJSON (builtins.readFile ./elixir-ls.json);
in
stdenv.mkDerivation rec {
name = "elixir-ls";
version = json.rev;
nativeBuildInputs = [ elixir hex git deps cacert ];
deps = fetchMixDeps {
name = "${name}-${version}";
inherit src;
sha256 = "1j7v56mfa087wi3x8kdcbqq0wsdiw284cwlccvxs1b60rypx5k55";
};
# refresh: nix-prefetch-git https://github.com/elixir-lsp/elixir-ls.git [--rev branchName | --rev sha]
src = fetchFromGitHub json;
dontStrip = true;
configurePhase = ''
runHook preConfigure
export MIX_ENV=prod
export HEX_OFFLINE=1
export HEX_HOME="$PWD/hex"
export MIX_HOME="$PWD"
export MIX_REBAR3="${rebar3}/bin/rebar3"
export REBAR_GLOBAL_CONFIG_DIR="$out/rebar3"
export REBAR_CACHE_DIR="$out/rebar3.cache"
cp --no-preserve=all -R ${deps} deps
mix deps.compile --no-deps-check
runHook postConfigure
'';
buildPhase = ''
runHook preBuild
mix do compile --no-deps-check, elixir_ls.release
runHook postBuild
'';
installPhase = ''
mkdir -p $out/bin
cp -Rv release $out/lib
# Prepare the wrapper script
substitute release/language_server.sh $out/bin/elixir-ls \
--replace 'exec "''${dir}/launch.sh"' "exec $out/lib/launch.sh"
chmod +x $out/bin/elixir-ls
# prepare the launcher
substituteInPlace $out/lib/launch.sh \
--replace "ERL_LIBS=\"\$SCRIPTPATH:\$ERL_LIBS\"" \
"ERL_LIBS=$out/lib:\$ERL_LIBS" \
--replace "elixir -e" "${elixir}/bin/elixir -e"
'';
}
And so far it works mostly flawlessly.
6 Likes
Apologies for my absence since the first post, lots of work stuff leaching my attention this week. Thanks to everyone who responded so far and I’ll try to catch up this weekend. 
1 Like
would you consider upstreaming at least fetchMixDeps and maybe elixir-ls as well? It looks great to me.
3 Likes
Linking the Status of lang2nix approaches discussion here because I think it’s pertinent.
Just to connect the dots:
1 Like