I’m trying to use buildGoModule to build a nix derivation for deployment of a local Go package (web service). Just running go build takes ~2 seconds to build everything. Running nix-build api.nix takes about 1 minute (only buildPhase, not counting checkPhase and onwards).
The problem is likely sqlite3 being a cgo package, for some reason not being cached and constantly recompiled.
Here are some pointers:
for every rebuild (even though I don’t touch any code), a new derivation path is being built by nix. This is probably a symptom of a root cause, that finally causes everything to be recompiled from scratch
Using both buildGoModule and basic mkDerivation aproach has the same behavior. Pasting both files below.
The majority of the time is wasted with clang (likely compiling sqlite3), but I don’t have great ideas how to determine which exact step is being wasteful.
vendorSha256 is provided correctly, which confuses me a bit - thought this would make caching work ok
Here’s a buildGoModule version
with import<nixpkgs>{};
let
api = (buildGoModule {
name = "api";
src = ./.;
vendorSha256 = "1728j55s0x92jyfx8a5dh5a371x33dj7zipz3p5nxm7jn3qvhswh";
checkPhase = ''
echo "not important for the demo"
'';
});
in
stdenv.mkDerivation {
name = "api";
# subPackages = [ "." ]; # doesn't make a difference
src = ./.;
buildInputs = [ api ];
installPhase = ''
mkdir -p $out/bin
cp ${api}/bin/api $out/bin/api
'';
}
Here’s a basic mkDerivation version
with import<nixpkgs>{};
stdenv.mkDerivation {
name = "api";
src = ./.;
excludes = [ ./ops ];
buildInputs = [ go age ];
configurePhase = ''
export GOPATH=$TMPDIR/go
export GOCACHE=$TMPDIR/share/go
export GOSUMDB=off
export GOPROXY=off
'';
buildPhase = ''
go build
'';
checkPhase = ''
echo "testing..."
'';
installPhase = ''
mkdir -p $out/bin
mv api $out/bin/api
echo "DONE"
'';
}
You should probably try to use buildGoModule with cleaned sources without the wrapper to make builds quicker.
As for this one, as far as I understand this is to remove let api = buildGoModule ... in mkDerivation and just use buildGoModule on a top level? In real life, I’m using this derivation to also decrypt secrets and run a stripe-mock server, then run a full suite of unit & integration tests in the checkPhase. Think i have no other choice, but maybe I’m getting this all wrong
Hm… just spent an hour trying to debug further and try a couple of suggested items. lib.cleanSource stripped a lot of unnecessary stuff, but still didn’t address the issue.
This means, that every time any file (including .nix files) is changed, buildGoModule recompiles everything again. I’ve removed the mkDerivation as well and just went with straight buildGoModule as suggested. Still no cigar.
with import<nixpkgs>{};
let
stripe = import ./test/stripe-mock.nix;
in
buildGoModule {
name = "api";
# src = lib.cleanSource (lib.sourceFilesBySuffices ./. [ ".go" ".mod" ".sum" ]); # this is more strict but doesn't load fixtures etc.
src = lib.cleanSource ./.;
vendorSha256 = "1728j55s0x92jyfx8a5dh5a371x33dj7zipz3p5nxm7jn3qvhswh";
checkPhase = ''
${stripe}/bin/stripe-mock &
# ... run tests, kill stripe-mock ...
echo "DONE"
exit 0
'';
installPhase = ''
mkdir -p $out/bin
cp bin/api $out/bin/api
'';
}
EDIT: It makes sense that the derivation needs to be rebuilt after touching nix files, but I’d expect (1) a different nix store path / new derivation, and (2) a no-op for everything else - there should be no clang or go building going on.
This is as expected. The nix files are part of your source, and if anything in your source changes, everything depending on that source needs to get recompiled.
Due to how golang builds, you can’t even split dependency and code building into separate derivations like rust does. You simply have to live with the full rebuild each time.
They should not, they’re not necessarily during compilation. The only reason why changing nix source should trigger a recompilation is if derivations in the graph have actually changed, that is, the evaluated expressions, not necessarily the source itself.
I believe it’s much easier to use whatever language toolset during development, nix cannot easily benefit from incremental compilation. I only run a nix build before committing for good measure to make sure everything still works
Yep thanks, that’s what I’m doing right now. Keeping everything vanilla and building/testing with the go tool. But still for deployment, something that takes 2 seconds taking 1 minute is just in the way.
Cleans out VCS dirs (.git, .hg, …), build artifacts (.o, .so), swap files and anything ending with ~, symlinks, type = directory, and result.
This effectively means yes - .nix files are a part of src. And they should be since they define how stuff is built. What shouldn’t be happening is once build flags are computed and deps fetched, deps themselves shouldn’t be rebuilt over and over again. Probably the product itself (non-nix) shouldn’t either.
Sorry to revive the topic but let’s hope this helps some poor soul…
There is a solution building sqlite as a static library (libsqlite.a) and passing the tag “libsqlite3” to Go so the library mattn/go-sqlite3 links against it. I’m vendoring Go dependencies (go mod vendor)