# Speeding up Rust app packaging
**URL:** https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384
**Category:** Help
**Created:** [March 23, 2020, 5:40pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384 "2020-03-23T17:40:28Z")
**Posts on this page:** 14
**Page:** 1
### Author:  [@savannidgerinel](https://discourse.nixos.org/u/savannidgerinel)
#### Post date: [March 23, 2020, 5:40pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/1 "2020-03-23T17:40:29Z")
I have a Rust app that I’m ready to get packaged up. I have a derivation that builds it, but the final product isn’t quite right. I’ve been working on the post-install script to try to find the magic combination of things that need to be wrapped around the executable…
but it takes 20-ish minutes with every one of my cores to build this app using `rustPlatform.buildRustPackage` (which is _far_ more than it takes with just Cargo), plus it does a fresh rebuild on every install, even though my only change is to the postInstall script.
So, I’m thinking to speed things up a lot by having one derivation for the package itself, and then another derivation that wraps the final executable in whatever scripts that I need to make it run right.
With that, I have `crate.nix`:
```auto
{ pkgs,
fetchFromGitHub ? pkgs.fetchFromGitHub,
stdenv ? pkgs.stdenv,
rustPlatform ? pkgs.rustPlatform,
}:
let
in rustPlatform.buildRustPackage rec {
...
}
```
And then I have `default.nix`:
```auto
{ pkgs,
fetchFromGitHub ? pkgs.fetchFromGitHub,
stdenv ? pkgs.stdenv,
rustPlatform ? pkgs.rustPlatform,
}:
let crate = import ./crate.nix { inherit pkgs fetchFromGitHub stdenv rustPlatform; };
in stdenv.mkDerivation rec {
name = "fitnesstrax-${version}";
version = "0.0.5-pre3";
buildInputs = [
crate
pkgs.makeWrapper
pkgs.gsettings-desktop-schemas
];
postInstall = ''
makeWrapper $fitnesstrax/bin/fitnesstrax-gtk $out/bin/fitnesstrax --set GSETTINGS_SCHEMA_DIR $out/share --set XDG_DATA_DIRS=$GSETTINGS_SCHEMAS_PATH:$XDG_DATA_DIRS
'';
}
```
But… `mkDerivation` requires a `src` path, which isn’t relevant in this case.
- Should I be using `mkDerivation`, or something like `symlinkJoin`?
- Is `postInstall` the right way to go here, or should I instead be creating a launcher script, like… `launcher = 'makeWrapper $fitnesstrax/bin/fitnesstrax-gtk $out/bin/fitnesstrax --set ...'`?
---
### Author:  [@danbst](https://discourse.nixos.org/u/danbst)
#### Post date: [March 23, 2020, 7:12pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/2 "2020-03-23T19:12:43Z")
This is something I’d like to have generic solution for. Your approach is fine
- you can set `src = crate;`
- but you also can also use dedicated `pkgs.runCommand`
- `symlinkJoin` is fine if `ln -s` approach can’t be applied
---
### Author:  [@shanesveller](https://discourse.nixos.org/u/shanesveller)
#### Post date: [March 23, 2020, 7:18pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/3 "2020-03-23T19:18:21Z")
Two community projects that attempt to behave better with regards to incremental builds:
> **[GitHub - nix-community/crate2nix: rebuild only changed crates in CI with...](https://github.com/nix-community/crate2nix)**
>
> rebuild only changed crates in CI with crate2nix and nix - GitHub - nix-community/crate2nix: rebuild only changed crates in CI with crate2nix and nix
> **[GitHub - nix-community/naersk: Build Rust projects in Nix - no configuration,...](https://github.com/nix-community/naersk)**
>
> Build Rust projects in Nix - no configuration, no code generation, no IFD, sandbox friendly. \[maintainer=@AxelSilverdew\] - GitHub - nix-community/naersk: Build Rust projects in Nix - no configurati...
YMMV but I find the former gels slightly more with my mindset and workflows, and I’m definitely feeling the pain of upstream Nix support while trying to work in my first non-toy Rust project. I’m using the `kube`, `k8s-openapi`, `prometheus`, and `actix-web` crates in practice. Crate2nix handles that very satisfactorily, up to and including the ability to generate Docker images out of it using the `nixpkgs` facilities for this.
There are a few more options cited in the two READMEs that I don’t have any first-hand experience with.
---
### Author:  [@savannidgerinel](https://discourse.nixos.org/u/savannidgerinel)
#### Post date: [March 23, 2020, 7:25pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/4 "2020-03-23T19:25:27Z")
I actually got some advice that .nix files generate from the likes of Carnix (and it looks like Crate2Nix) generally won’t be accepted into nixpkgs because they’re too large to be properly reviewed. That’s why I’m using buildRustCrate.
naersk looks interesting, though.
---
### Author:  [@danieldk](https://discourse.nixos.org/u/danieldk)
#### Post date: [March 23, 2020, 8:53pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/5 "2020-03-23T20:53:18Z")
That’s interesting to hear, because there is a small number of packages in Nix that use buildRustCrate. (Maybe 2 or 3.)
---
### Author:  [@bhipple](https://discourse.nixos.org/u/bhipple)
#### Post date: [March 24, 2020, 1:08am UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/6 "2020-03-24T01:08:13Z")
One reason why it takes longer than a clean cargo build is that Nix is currently building/installing the release binary target, but it’s running the tests in debug mode, so it ends up compiling all the libraries twice.
There’s work underway to try to address this, which you might be interested in trying out:
>
>
> \###### Motivation for this change
>
> \ :warning: \*\*Caution:\*\* this is more o…r less an experiment and I'm not sure if that has implications on any leaf-packages since I just tested a few packages like rustup or cargo-audit with it, so this needs some review and testing before becoming mergable.\
>
> As noted \[later on\](https://github.com/nixos/nixpkgs/pull/82342#issuecomment-626018989), I consider this rather usable after several iterations of reviews and tests, however we need more extensive testing before merging this to upstream.
>
> \----
>
> When running \`cargo test --release\`, the artifacts from \`buildPhase\`
> will be reused here. Previously, most of the stuff had to be recompiled
> without optimizations.
>
> \###### Things done
>
>
>
> \- \[x\] Tested using sandboxing (\[nix.useSandbox\](http://nixos.org/nixos/manual/options.html#opt-nix.useSandbox) on NixOS, or option \`sandbox\` in \[\`nix.conf\`\](http://nixos.org/nix/manual/#sec-conf-file) on non-NixOS linux)
> \- Built on platform(s)
> - \[x\] NixOS
> - \[\] macOS
> - \[\] other Linux distributions
> \- \[\] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside \[nixos/tests\](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
> \- \[\] Tested compilation of all pkgs that depend on this change using \`nix-shell -p nixpkgs-review --run "nixpkgs-review wip"\`
> \- \[\] Tested execution of all binary files (usually in \`./result/bin/\`)
> \- \[\] Determined the impact on package closure size (by running \`nix path-info -S\` before and after)
> \- \[\] Ensured that relevant documentation is up to date
> \- \[x\] Fits \[CONTRIBUTING.md\](https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md).
---
### Author:  [@danieldk](https://discourse.nixos.org/u/danieldk)
#### Post date: [March 24, 2020, 7:01am UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/7 "2020-03-24T07:01:44Z")
> [@bhipple](#):
>
> There’s work underway to try to address this, which you might be interested in trying out:
I wonder if this is actually a good idea, debug doesn’t just compile with fewer optimizations, it also turns on integer overflow checks and debug assertions (which are used throughout the standard library and crates to do more expensive checks).
Edit: maybe I should raise my concern in the issue?
---
### Author:  [@Ma27](https://discourse.nixos.org/u/Ma27)
#### Post date: [March 24, 2020, 3:19pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/8 "2020-03-24T15:19:18Z")
> Edit: maybe I should raise my concern in the issue?
Yes, may be better (I’m not an active discourse reader).
> I wonder if this is actually a good idea, debug doesn’t just compile with fewer optimizations, it also turns on integer overflow checks and debug assertions (which are used throughout the standard library and crates to do more expensive checks).
Hmm, I see. But isn’t this an issue we have with every rust-package with `doCheck = false`?
Also, those sort of issues should be something catched by upstream before publishing a release, right? I mean during development, you’d usually have `debug` enabled I guess.
---
### Author:  [@symphorien](https://discourse.nixos.org/u/symphorien)
#### Post date: [March 24, 2020, 3:50pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/9 "2020-03-24T15:50:11Z")
My understanding is that the goal of running the test suite as part of a derivation is to check that the patching we do with our uncommon toolchain (hardcoded paths, patchelf, etc.) does not break the software too much. I think this specific type of problems will surface as easily with a debug build or a release build. Problems with overflows should be caught upstream.
---
### Author:  [@savannidgerinel](https://discourse.nixos.org/u/savannidgerinel)
#### Post date: [March 25, 2020, 3:25am UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/10 "2020-03-25T03:25:29Z")
Thanks for this. Not trying it out on this derivation, at least not right now, since I’ve been running as hard as I can to finish a release and get this app out and available to people.
But, in starting to look at PRs (I had no idea how many open PRs there were), I actually came across this, so it will be one of the things that I attempt to review and test.
---
### Author:  [@kolloch](https://discourse.nixos.org/u/kolloch)
#### Post date: [April 29, 2020, 11:35am UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/11 "2020-04-29T11:35:55Z")
> [@savannidgerinel](#):
>
> I actually got some advice that .nix files generate from the likes of Carnix (and it looks like Crate2Nix) generally won’t be accepted into nixpkgs because they’re too large to be properly reviewed. That’s why I’m using buildRustCrate.
It would be interesting to know who gave you this advice.
I am the maintainer of crate2nix and I actually got encouraging noises about making crate2nix ready for becoming the standard solution for nixpkgs. Currently, the support for out-of-tree sources in crate2nix relies on “import from derivation”. I have dabbled with support for out-of-tree sources without that lately with the intention of making crate2nix a prime tool for nixpkgs.
Therefore, I’d love to know who I need to convince. 😉
---
### Author:  [@savannidgerinel](https://discourse.nixos.org/u/savannidgerinel)
#### Post date: [April 29, 2020, 4:05pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/12 "2020-04-29T16:05:37Z")
_ugh_ I don’t remember any more. It was somebody in one of the Nix channels, but I don’t remember whether it was the #nixos-chat channel or one of the smaller ones.
And now that I’ve gotten my app into the repository 🥳, I’m reluctant to change it.😑
---
### Author:  [@kolloch](https://discourse.nixos.org/u/kolloch)
#### Post date: [May 1, 2020, 6:28pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/13 "2020-05-01T18:28:01Z")
Thanks for getting back to me, nevertheless 🙂
---
### Author:  [@zimbatm](https://discourse.nixos.org/u/zimbatm)
#### Post date: [May 1, 2020, 7:13pm UTC](https://discourse.nixos.org/t/speeding-up-rust-app-packaging/6384/14 "2020-05-01T19:13:50Z")
Here is an issue by the Nix author that wants to restrict fixed-output derivations: [Restrict fixed-output derivations · Issue #2270 · NixOS/nix · GitHub](https://github.com/NixOS/nix/issues/2270)
We also have @lewo who is working with Software Heritage to mirror all of our sources, which is made more difficult when fetching all the sources as one derivation.
On the other hand, listing all the dependencies in nixpkgs can make the packages quite large. Especially for nodejs applications where even simple applications tend to have thousands of dependencies. This directly impacts how long you have to wait on channel updates or when using projects with pinned nixpkgs versions.
It’s a trade-off and there is no clear outcome yet.