Rust package appears to need network during build

Help
1

Hello,

I’m currently trying to build Mangatan on NixOS. I have this nix file so far:

{
  fetchFromGitHub,
  lib,
  rustPlatform,
  openssl,
  pkg-config,
  libglibutil,
  gtk3,
  libappindicator-gtk3,
  librsvg,
  fuse,
  nasm,
  gcc,
  cmake,
  perl,
  xdo,
}:
rustPlatform.buildRustPackage (finalAttrs: {
  pname = "mangatan";
  version = "1.3.0";

  src = fetchFromGitHub {
    owner = "KolbyML";
    repo = "Mangatan";
    rev = "v1.3.0";
    hash = "sha256-oZ1+aWscFqPdBttvPBk1tLne9SKhCdaTT6uwnbyDt6I=";
  };

  cargoHash = "sha256-8wWg4LO47aRGsCIcM2hgJ8KtFN7uxFqvGXhqCCW/M+w=";

  nativeBuildInputs = [pkg-config gcc cmake perl];
  buildInputs = [openssl libglibutil gtk3 libappindicator-gtk3 librsvg fuse nasm xdo];

  meta = {
    description = "A 1 click solution for reading Manga with OCR'ed textboxs, works seamless with Yomitan";
    homepage = "https://github.com/KolbyML/Mangatan";
    license = lib.licenses.mit;
    maintainers = [];
  };

  installPhase = ''
    runHook preInstall
    mkdir -p $out/bin
    cp mangatan $out/bin
    runHook postInstall
  '';
})

When trying to build, I get this error:

error: failed to run custom build command for `lindera-unidic v0.43.3`

Caused by:
  process didn't exit successfully: `/build/source/target/release/build/lindera-unidic-3aeb8cd85c800f36/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-changed=build.rs
  cargo:rerun-if-changed=Cargo.toml
  cargo::rustc-env=LINDERA_WORKDIR=/build/source/target/aarch64-unknown-linux-gnu/release/build/lindera-unidic-cd9022d10fc41819/out

  --- stderr
  Error: "Failed to download a valid file from all sources"

Looking at the source of the lindera-unidic crate, it appears that it needs network access during build, but Nix doesn’t allow that. How do I solve this?

2

In general, you cannot “solve” this; nix prevents downloads during builds precisely to show you broken software that relies on downloads at build-time, since those sources cannot be adequately tracked, making them very difficult to build reproducibly (you’d need to set up some kind of proxy that presents cached sources and probably MITM https connections).

So, for any software that insists on this, you need a project-specific workaround. The library you’re trying to build already has a solution of some sort: Download dictionary during building · Issue #522 · lindera/lindera · GitHub

My personal preference is to just not use software that requires external dependencies like this at build time, but I don’t know how well-standardized this particular type of external dependency is.

3

I see, thank you so much for finding this!

Another noob question: How do I make nix execute the commands mentioned in the solution? I. e. how do I append something to a phase (e. g. configPhase, buildPhase) and which phase would be suitable for this?

4

As far as I can tell, you wouldn’t use any extra commands in any phases, but instead add a LINDERA_CACHE attribute (env.LINDERA_CACHE if you are using structured attrs) which you set to a fetchTarball call that pulls the tarball and extracts it.

5

Not quite, given the commands there it wants the raw, non-extracted tarball. It also wants it in a specific subdirectory, so you’re going to have to do some symlinkJoin shenanigans. Even then, YMMV, it’s quite possible it wants read-write access or other minor annoyances. It’ll take a bit of fiddling.

It’s possible it’s easier to just bind a fetchTarball to a variable in a let and then put it in the correct place with mkdir and cp in a preFixup or something.

6

How would I do that? I’ve got the fetchTarball in a let but I’m lost on how to use that in preFixup. Also, I’ve noticed that the package I’m trying to build is using lindera version 0.43 ( Mangatan/crates/yomitan-server/Cargo.toml at ca1c85fd16ff4cba0c15c502cc86c32777d14d40 · KolbyML/Mangatan · GitHub ), but the author’s reply on the issue implies one needs version 0.45. How do I override this?

7

Theoretically something like:

preFixup = ''
  export LINDERA_CACHE=lindera_cache
  mkdir -p "$LINDERA_CACHE/0.45"
  cp ${var} "$LINDERA_CACHE/0.45/mecab-ipadic-2.7.0-20070801.tar.gz"
'';

Though fetchTarball auto-extracts to my knowledge, so that shouldn’t work. You’ll need pkgs.fetchurl.

I’m also not certain that the rust derivation tools use this phase, and that they don’t override preFixup.

I’m also unsure if the export is needed, or if it’s even an issue; depends on how buildRustPackage calls out to subprocesses.

It’s something along those lines, anyway. You may want to read the basic packaging and language tutorials.

This might be harder still. You can normally use patchPhase to edit your source before the build, but the rust tooling creates a separate derivation for your input crates. It’d anyway not work because of Cargo.lock.

At that point you’ll have to read the docs for buildRustPackage and provide your own cargolock - this isn’t a trivial package :wink: