Go derivation for filestash not playing ball

[nixinator]

Trying to write a derivation for GitHub - mickael-kerjean/filestash: 🦄 A modern web client for SFTP, S3, FTP, WebDAV, Git, Minio, LDAP, CalDAV, CardDAV, Mysql, Backblaze, ...

I’m using the the nix function buildGoModule

{ buildGoModule, fetchFromGitHub, lib }:

buildGoModule rec {
  pname = "filestash";
  version = "0.4";

  src = fetchFromGitHub {
    owner = "mickael-kerjean";
    repo = "filestash";
    rev = "v${version}";
    sha256 = "0zi8hp3w1d9zsmn0rzfzw7jj38rdy915vqkwyv2gq81jz2jf4h4c";
  };

  vendorSha256 = "0pnd89iqdj3f719xf4iy5r04n51d0rrrf0qb2zjirpw7vh7g82i9";
  subPackages = [ "." ];
  runVend = true;

  meta = with lib; {
    description = "A Dropbox-like file manager that let you manage your data anywhere it is located";
    homepage = https://github.com/mickael-kerjean/filestash;
    license = licenses.mit;
    maintainers = with maintainers; [ nobody  ];
    platforms = platforms.linux;
  };
}

You can run it with

nix-build -E "with import <nixpkgs> {}; callPackage ./default.nix {}"

Build fails with

go: cannot find main module; see 'go help modules'

All i can see is that main.go is nested inside ./server/main.go

The repo’s that buildGoModule seem to always have main.go in ./

Is there a flag i can use to express this, or perhaps go build magic?

Or is the problem something more subtle.

[nixinator]

debugging with

nix-shell -E "with import <nixpkgs> {}; callPackage ../default.nix {}"
genericBuild

Ah, I think the branch tag i’m checking out doesn’t support go modules…which is probably the reason why it can’t be built with goBuildModule.

[nixinator]

Yep, fetching from master get me a lot further. DOH! . The pleasure of go have 5 different build system in it’s evolution…maybe even 6

Just now how to work out what PKG_CONFIG_PATH needs to be set up with and patch the make file which hardcoded to

PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/ 

which obviously in not nix friendly.

[nixinator]

Well, I made some good progress.

Seems the package build a static version of libvips - which provides libresize_linux_amd64.a , using a shonky brittle shell scripts…and guess what, you can build everything with a docker container…exactly some of the problems that nix tries to solve :-).

The classic case of, Why was docker invented…‘so tensor flow could compile’.

so close, but …so far . enough for now! was a good bit of ‘fun’.

{ buildGoModule, fetchFromGitHub, lib, pkg-config, glib, vips }:

buildGoModule rec {
  pname = "filestash";
  version = "0.4";

  src = fetchFromGitHub {
    owner = "mickael-kerjean";
    repo = "filestash";
    rev = "030b5b0c941e06bb31d7baedc4185340e799cad7";
    sha256 = "1nqdaxyf9inxdvglmi661c0ph8vl5ca3xgcic6y6kcybgzg1wady";
  };

  nativeBuildInputs = [ pkg-config ];
   propagatedBuildInputs = [
    glib vips
  ];

  BuildInputs = [ vips];

  #vendorSha256 = "1pnd89iqdj3f719xf4iy5r04n51d0rrrf0qb2zjirpw7vh7g82i9";
  vendorSha256 = null;
  subPackages = [ "server/." ];
  #runVend = true;

  meta = with lib; {
    description = "A Dropbox-like file manager that let you manage your data anywhere it is located";
    homepage = https://github.com/mickael-kerjean/filestash;
    license = licenses.mit;
    maintainers = with maintainers; [ nobody  ];
    platforms = platforms.linux;
  };
}

/nix/store/10nmk0mlhy9nf3lfrq8q07k0510bgnnf-binutils-2.31.1/bin/ld: cannot find -l:libresize_linux_amd64.a

[nixinator]

Okay… well it works!. The original had lots of shells scripts to extract all the vips and libraw libraries into his custom libraries to build libresize.a & libtranscode.a , or they just curl a precompiled .a’s from his website!! :-). Seems that they had trouble getting these libraries to compile due to dependices etc etc…same old story…

I had to basically tell the go’s CGO to link in -lvips & -lraw (it seems libraw linker directive is not -llibraw…eeeeeeekkkkkk ARRRRRGGGHHHH LOL).

server/plugin/plg_image_light/lib_resize_linux_amd64.go
3:// #cgo LDFLAGS: -lm -lgmodule-2.0 -lgobject-2.0 -lglib-2.0 -ldl -L./deps -l:libresize_linux_amd64.a -lvips

server/plugin/plg_image_light/lib_transcode_linux_amd64.go
3:// #cgo LDFLAGS: -lm -lpthread -L./deps -l:libtranscode_linux_amd64.a -lraw

I don’t there is any way i regenerate these cgo flags in these files, I don’t think they are generated by automagically.,…some cgo LDFLAGS are…

Now i need to patch files with nix…and should be in the home strait (jacket)…?

the front end of this app is in node.js, so i’m probably compleltly doomed!

[nurelin]

Hello,

Have you been able to succeed in packaging this application ?
I am interested too, if you need help.

[nixinator]

life got in the way, i might finish it of at camp if i get chance,

it looks like quite a cool app.

[nurelin]

I got it to build in nixos.
But unfortunately, it seems that the binary expects to be able to write in its folder.
See the usage of GetCurrentDir() in the project.

I have put the whole derivation here.
But I do not see how to make it work so I will probably stop there.

[nixinator]

yeah, that would do it, like many things in linux/unix, the expect that things are in the usual places, and that those places are read/write, unlike the /nix/store.

It’s going to have to fixed upstream, or patch in the source code in the derivation. Either-way, it’s a bit of work.

[pinpox]

Would be interested in using this. I’ll give your derivation a try later

[pinpox]

Hey, small update:

@nurelin I got this PR merged in filestash: Add Working directory override by pinpox · Pull Request #403 · mickael-kerjean/filestash · GitHub
which adds a WORK_DIR environment variable. With that set to a writable directory, your issue should be resolved. GetCurrentDIR() will be overriden by that value, making the application able to use other directories then where the binary is.

Would you be interested in giving it a try?

Just for reference, this is the complete GetCurrentDir() function in the applications code base now:

func GetCurrentDir() string {
	if MOCK_CURRENT_DIR != "" {
		return MOCK_CURRENT_DIR
	}
	if os.Getenv("WORK_DIR") != "" {
		return os.Getenv("WORK_DIR")
	}
	ex, _ := os.Executable()
	return filepath.Dir(ex)
}

[nurelin]

Sorry, I have put that aside for now.
Feel free to continue.

[matthewcroughan]

@pinpox @nurelin I got a bit further, but instead of using node2nix for the conversion, I decided to use dream2nix and ran into the following issue. It would be wonderful if you could help debug or offer some insight on the issue, which seems to be to do with webpack loaders in dream2nix, and then we’ll have filestash almost fully packaged.

React "unexpected token", missing loader?

[nixinator]

@matthewcroughan , we really need to get nix bounties running…

in the same vein as https://gitcoin.co/

i thinking it could be specific for the nix ecosystem for paying or tipping nix commits, etc etc etc.

Something can do a number of payment systems too and be to payout out via contract once a nixpkgs/nix/nixos PR is merged, or accepted by all parties.

I wish i had time to do it, i wish i could just do

while true
do
git clone https://nixinator.minime
done

[pinpox]

Sorry, I don’t have the time to continue right now on this. I’ll stay subscribed here in case there is any progress, but can’t provide much help currently myself.

[nurelin]

I tried to play with your derivation but I do not know the javascript ecosystem enough to fix the problem.

[matthewcroughan]

@nurelin @pinpox I figured everything out and created a high quality flake with a NixOS Module, VM Test and CI here

GitHub - MatthewCroughan/filestash-nix: A flake for testing, building and...

I also created a PR here that would have it added to NixOS Search? if it were merged.

Add filestash-nix to the flakes index