Announcing Mixnix - Build Elixir projects with Nix

Announcements
1

This is the first step on a path to better support for Elixir and Erlang via Nix.
So far it’s been nearly impossible to build arbitrary projects with Nix, given that the versions of BEAM packages in nixpkgs are outdated and incomplete.
The current approach is also not sustainable, given the amount of changes happening on hex.pm.

The approach I’ve taking is similar to what moretea did with yarn2nix: given a lock file we want to create a derivation on the fly, and if IFD is an issue, create a nix-readable lockfile to speed up builds.
Also important was for me a bundix-like approach to per-package configuration and a preset of common hacks needed to get certain packages to build.

You can find the result at Michael Fellinger / mixnix · GitLab and give it a go.

I’d be especially thankful for suggestions of projects that are worth packaging for nixpkgs that have been ignored so far because it was too hard.

I’d also like to give a special thanks to all the people who’ve worked on the BEAM packaging support in nixpkgs. This wouldn’t have been possible without all their hard work that this is based on.

18 Likes
2

That’s cool! :+1:
It would probably be a good idea to also add mix-to-nix to the “Related Work” section and have some sort of comparison between the two.

3 Likes
3

Damn, I wish I’d found this earlier that’d have saved me a ton of work, but even after days of searching for anything related to Elixir and Nix this project didn’t show up for me.

I’ll have to take a closer look, but it seems nearly identical. MixNix has better support for git dependencies and a common override system but it lacks the fake hex registry (I’m just using a snapshot placed in the repo). MixNix also generates a .nix file for the lock so it’s possible to use it without import-from-derivation and is more suitable for inclusion of packages in nixpkgs.

The MixNix usage is also more similar to how Bundix and Yarn2nix work.

When I get some time I can do some better comparison and will include it in the docs. It’s just fun to see how similar the two solutions are, so I think we’re on to something here.

3 Likes
4

I am also checking out GitHub - transumption/mix-to-nix: Sandboxable mix.lock and rebar.lock Nix generator

5

I tried both, but none worked. My application is an almost standard
Phoenix application which I want to distribute with the new the new
mix release way.

mixToNix had issues with unpacking tar files downloaded from hex.pm.

mixnix had initially an issue with the format of mix.lock which
seems to have changed. Fixed that I now get an error like:

rror: while evaluating the attribute 'beamDeps' of the derivation 'journey' at /nix/store/kjq19d7gvq9c80gnsm5zzwz992sa739s-nixos-20.09pre217261.a2e06fc3423/nixos/lib/attrsets.nix:194:41:
while evaluating the attribute 'beamDeps' of the derivation 'bamboo' at /home/azazel/mixnix/nix/mix2nix.nix:233:13:
while evaluating the attribute 'beamDeps' of the derivation 'hackney-1.15.2' at /nix/store/kjq19d7gvq9c80gnsm5zzwz992sa739s-nixos-20.09pre217261.a2e06fc3423/nixos/pkgs/development/beam-modules/build-rebar3.nix:35:5:
while evaluating the attribute 'beamDeps' of the derivation 'certifi-2.5.1' at /nix/store/kjq19d7gvq9c80gnsm5zzwz992sa739s-nixos-20.09pre217261.a2e06fc3423/nixos/pkgs/development/beam-modules/build-rebar3.nix:35:5:
while evaluating anonymous function at /home/azazel/mixnix/nix/mix2nix.nix:215:32, called from undefined position:
attribute 'parse_trans' missing, at /home/azazel/mixnix/nix/mix2nix.nix:215:41

and here I stopped … @manveru if you want you can look at my fix
here Alberto / mixnix · GitLab . I’ve discovered the fixes
branch and I’ve tried it out as well, but without luck.

1 Like
6

The fact that mix or hex is trying to repair the lock file is indeed considered a bug by José Valim. I’ll dig this weekend into the source to check which components fault it is and submit a bug report in the appropriate repository.

Though I can’t help you with the other issue. I’ve a single mix project I build via nix, and that does only work in non sandbox mode as it does a lot of irregular stuff,which is beeing worked on upstream to get fixed.

1 Like
7

Uh? what are you referring to? Wrong thread?

8

This is what I am referring to.

9

Thanks for publishing the fix to your first problem, I had exactly the same problem.
I noticed that the second issue is caused by some entries not being included in mix.nix. I saw that all of these entries had an additional hash as a last column. I removed these hashes in vim with :%s/, "[a-f0-9]\{64\}"},/},/ and this made them appear in mix.nix!
I am not sure what these additional hashes are supposed to be used for. Is there a description of the mix.lock file format available anywhere?

10

The mix.lock files format is considered opaque. Mostly because each “downloader” can do it’s own thing.

The 2 hashes are one hash for the tarball, and one hash for the contents. This though is only done by the hex downloader.

Other downloaders might have other metadata or hashes in the file.

1 Like
11

Indeed, I encountered the same issue while trying to package Mobilizon. And I confirm your trick made appear the missing packages.
Sounds like a patch in mixnix would solve that.

12

And I come back with multiple issues:

  • Apparently, MixNix uses builtins.fetchGit sometimes for Git dependencies, but it does not set properly the ref, meaning that if the dependency is not using master or something akin to, and a feature branch, or whatever (like Mobilizon does :-))))))))), it will fail. The fix is to add ref = XXX in the mix.nix for example.

  • Then, dependencies like https://github.com/CrowdHailer/git_status/blob/efb88e993813996946e00e92ef31c1eb86aa49ff/lib/git_status.ex will fail with probability 1 because git is not available in the PATH, I believe. I’m not sold on what would be the proper fix? Requiring that this package does not fail if git is not available or adding an overlay for this kind of exception to make git available.

13

Try seeing if @hauleth’s Nix code here is more helpful - I haven’t had an opportunity to package something non-trivial with it yet:

14

Will give it a try, I’m stuck with MixNix, after many adhoc patches.
With the following error:

===> Plugin rebar3_hex not available. It will not be used.
===> Verifying dependencies...
===> Package not found in any repo: cowboy ~> 2.7

With cowboy-telemetry package. (Here’s my patches: Raito Bezarius / mixnix · GitLab)

15

It fails immediately with

17:49:44.708 [error] No configuration file found
17:49:44.708 [error] No configuration file found
** (Mix) The task "compile.phoenix" could not be found. Did you mean "compile.elixir"?
** (Mix) The task "compile.phoenix" could not be found. Did you mean "compile.elixir"?

Unsure of what I’m doing wrong:

{ buildMix, fetchgit }:
let
  src = fetchgit {
    url = "https://framagit.org/framasoft/mobilizon";
    rev = "7c43d52b20a61a4db30536e4d3bfda398a8c6603";
    sha256 = "0v8rwd9i967mpi69adii8zjmh7v4srkjsfjxrnnphszp9k6yvl1d";
  };
in
  buildMix {
    name = "mobilizon";
    version = "1.0.0";
    inherit src;
  }

is my expression.

16

I don’t have time to run this down any further but your custom compilation step here is what is throwing that approach off:

I don’t recognize it (mix compile.phoenix etc.) as a common practice - perhaps it is from some historical code, something bespoke to your project, etc.

17

To be fair, I have no idea what it does (and it’s not even my project :D…), but thanks for the help!
It looks like down the road, it uses network access at some point to download locales… :confused:

18

@RaitoBezarius if you are still interested, I have managed to get the Elixir part of Mobilizon to build (see discussion here). I still need to get the yarn part to build, but I have no knowledge of yarn, so this might prove to be more difficult.

19

I can give you a hand for the Yarn part, indeed!
You can DM me here, or over IRC (Raito_Bezarius@freenode)

20

Finally, after loads of trial and errors, and loads of help from everyone who contributed to buildMix: fix: initial try by happysalada · Pull Request #112477 · NixOS/nixpkgs · GitHub, I pushed a draft PR for Mobilizon here: https://github.com/NixOS/nixpkgs/pull/119132 :tada:

In the end I managed to fix the Yarn issue by patching the lock file. To my understanding, it was because yarn2nix doesn’t support the “resolutions” field in the project file.

Thank you @RaitoBezarius for offering your help, I really appreciate it.

If anyone is interested in the Mobilizon nixpkgs derivation, I’d be more than happy to add you as a maintainer.

3 Likes