Import problems creating a new package (45Drive's autotier)

CafeHaine · September 27, 2022, 11:30am

Hi everyone!

I’m trying to build my first packages for 45Drive’s autotier (a tiered fuse filesystem), and I’m getting stuck trying to fix an import to fuse.

When building the autotier package I get the following output:

Summary

[cafehaine@kgenix:~/Documents/Prog/nixpkgs]$ nix-build -A autotier
this derivation will be built:
  /nix/store/1kjv5j8dqkqcibcm3z8vm8majl4h871g-autotier-1.2.0.drv
building '/nix/store/1kjv5j8dqkqcibcm3z8vm8majl4h871g-autotier-1.2.0.drv'...
unpacking sources
unpacking source archive /nix/store/7iq1qjfxigg32bv3n1zl8iypamk9si1w-source
source root is source
patching sources
configuring
no configure script, doing nothing
building
build flags: SHELL=/nix/store/p7bpdnxqd3i5hwm92mrscf7mvxk66404-bash-5.1-p16/bin/bash
  CC build/autotierfs/config.o
  CC build/autotierfs/autotierfs.o
  CC build/autotierfs/TierEngine/components/mutex.o
  CC build/autotierfs/TierEngine/components/tiering.o
  CC build/autotierfs/TierEngine/components/adhoc.o
  CC build/autotierfs/TierEngine/components/database.o
  CC build/autotierfs/TierEngine/components/base.o
  CC build/autotierfs/TierEngine/components/sleep.o
  CC build/autotierfs/TierEngine/TierEngine.o
  CC build/autotierfs/fuseOps/link.o
In file included from src/impl/autotierfs/fuseOps/link.cpp:24:
src/incl/fuseOps.hpp:31:10: fatal error: fuse.h: No such file or directory
   31 | #include <fuse.h>
      |          ^~~~~~~~
compilation terminated.
make: *** [makefile:51: build/autotierfs/fuseOps/link.o] Error 1
error: builder for '/nix/store/1kjv5j8dqkqcibcm3z8vm8majl4h871g-autotier-1.2.0.drv' failed with exit code 2;
       last 10 log lines:
       >   CC build/autotierfs/TierEngine/components/base.o
       >   CC build/autotierfs/TierEngine/components/sleep.o
       >   CC build/autotierfs/TierEngine/TierEngine.o
       >   CC build/autotierfs/fuseOps/link.o
       > In file included from src/impl/autotierfs/fuseOps/link.cpp:24:
       > src/incl/fuseOps.hpp:31:10: fatal error: fuse.h: No such file or directory
       >    31 | #include <fuse.h>
       >       |          ^~~~~~~~
       > compilation terminated.
       > make: *** [makefile:51: build/autotierfs/fuseOps/link.o] Error 1
       For full logs, run 'nix log /nix/store/1kjv5j8dqkqcibcm3z8vm8majl4h871g-autotier-1.2.0.drv'.

The source for my packages is available here: https://github.com/cafehaine/nixpkgs

Thanks a lot for your time!

danielbarter · September 27, 2022, 4:53pm

looks like you need to include fuse as a build input to your derivation. the fuse header is included in the fuse package:

🥥 nix repl
Welcome to Nix 2.8.1. Type :? for help.

nix-repl> :l <nixpkgs>
Added 16534 variables.

nix-repl> fuse.outPath
"/nix/store/mz1267svxc1y5mfjzvk7lr98kl97jdfk-fuse-2.9.9"

nix-repl> :q
🥥 ls /nix/store/mz1267svxc1y5mfjzvk7lr98kl97jdfk-fuse-2.9.9/include
fuse  fuse.h  ulockmgr.h
🥥

Can you post the derivation you are trying to build?

CafeHaine · September 28, 2022, 9:21am

Sorry, I realise my first post lacked a lot of informations.

Here are the derivations I’ve written:

pkgs/development/libraries/lib45d/default.nix

{ lib, stdenv, fetchFromGitHub}:

stdenv.mkDerivation rec {
  pname = "lib45d";
  version = "0.3.6";

  src = fetchFromGitHub {
    owner = "45Drives";
    repo = "lib45d";
    sha256 = "sha256-42xB30Iu2WxNrBxomVBKd/uyIRt27y/Y1ah5mckOrc0=";
    rev = "v${version}";
  };

  installFlags = [ "LIB_PREFIX=/lib" "INCLUDE_PREFIX=/include" "DESTDIR=\${out}" "DEVEL=TRUE" ];

  meta = with lib; {
    description = "45Drives C++ Library";
    homepage = "https://github.com/45Drives/lib45d";
    license = with licenses; [ gpl3 ];
    platforms = platforms.unix;
  };
}

pkgs/tools/filesystems/autotier/default.nix

{ lib, stdenv, fetchFromGitHub
, fuse3
, lib45d
, boost
}:

stdenv.mkDerivation rec {
  pname = "autotier";
  version = "1.2.0";

  src = fetchFromGitHub {
    owner = "45Drives";
    repo = "autotier";
    sha256 = "sha256-XNM82yYwh3xapj3Euy++v0eKR1RVru9idOJzuKWBfIo=";
    rev = "v${version}";
    fetchSubmodules = true;
  };

  # Fix missing fstream import
  preBuild = ''
    sed '/<sstream>/a #include <fstream>' -i src/impl/autotierfs/config.cpp
  '';

  buildInputs = [ fuse3 lib45d boost ];

  installFlags = [ "DESTDIR=\${out}" ];

  meta = with lib; {
    description = "A multi tiered storage solution";
    longDescription = ''
      A passthrough FUSE filesystem that intelligently moves files between
      storage tiers based on frequency of use, file age, and tier fullness.
    '';
    homepage = "https://github.com/45Drives/autotier";
    license = with licenses; [ gpl3 ];
    platforms = platforms.unix;
  };
}

The autotier derivation doesn’t detect the fuse.h header when I’m using the fuse3 package.

fuse3

[cafehaine@kgenix:~/Documents/Prog/nixpkgs]$ nix-build -A autotier
this derivation will be built:
  /nix/store/1kjv5j8dqkqcibcm3z8vm8majl4h871g-autotier-1.2.0.drv
building '/nix/store/1kjv5j8dqkqcibcm3z8vm8majl4h871g-autotier-1.2.0.drv'...
unpacking sources
unpacking source archive /nix/store/7iq1qjfxigg32bv3n1zl8iypamk9si1w-source
source root is source
patching sources
configuring
no configure script, doing nothing
building
build flags: SHELL=/nix/store/p7bpdnxqd3i5hwm92mrscf7mvxk66404-bash-5.1-p16/bin/bash
  CC build/autotierfs/config.o
  CC build/autotierfs/autotierfs.o
  CC build/autotierfs/TierEngine/components/mutex.o
  CC build/autotierfs/TierEngine/components/tiering.o
  CC build/autotierfs/TierEngine/components/adhoc.o
  CC build/autotierfs/TierEngine/components/database.o
  CC build/autotierfs/TierEngine/components/base.o
  CC build/autotierfs/TierEngine/components/sleep.o
  CC build/autotierfs/TierEngine/TierEngine.o
  CC build/autotierfs/fuseOps/link.o
In file included from src/impl/autotierfs/fuseOps/link.cpp:24:
src/incl/fuseOps.hpp:31:10: fatal error: fuse.h: No such file or directory
   31 | #include <fuse.h>
      |          ^~~~~~~~
compilation terminated.
make: *** [makefile:51: build/autotierfs/fuseOps/link.o] Error 1
error: builder for '/nix/store/1kjv5j8dqkqcibcm3z8vm8majl4h871g-autotier-1.2.0.drv' failed with exit code 2;
       last 10 log lines:
       >   CC build/autotierfs/TierEngine/components/base.o
       >   CC build/autotierfs/TierEngine/components/sleep.o
       >   CC build/autotierfs/TierEngine/TierEngine.o
       >   CC build/autotierfs/fuseOps/link.o
       > In file included from src/impl/autotierfs/fuseOps/link.cpp:24:
       > src/incl/fuseOps.hpp:31:10: fatal error: fuse.h: No such file or directory
       >    31 | #include <fuse.h>
       >       |          ^~~~~~~~
       > compilation terminated.
       > make: *** [makefile:51: build/autotierfs/fuseOps/link.o] Error 1
       For full logs, run 'nix log /nix/store/1kjv5j8dqkqcibcm3z8vm8majl4h871g-autotier-1.2.0.drv'.

When I’m using the fuse package instead of fuse3, the fuse.h file is detected, but an enum in fuse.h isn’t properly detected by g++

fuse

[cafehaine@kgenix:~/Documents/Prog/nixpkgs]$ nix-build -A autotier
this derivation will be built:
  /nix/store/n6jk5wpcdbbs7yzfxccis6fiih2v2fbp-autotier-1.2.0.drv
building '/nix/store/n6jk5wpcdbbs7yzfxccis6fiih2v2fbp-autotier-1.2.0.drv'...
unpacking sources
unpacking source archive /nix/store/7iq1qjfxigg32bv3n1zl8iypamk9si1w-source
source root is source
patching sources
configuring
no configure script, doing nothing
building
build flags: SHELL=/nix/store/p7bpdnxqd3i5hwm92mrscf7mvxk66404-bash-5.1-p16/bin/bash
  CC build/autotierfs/config.o
  CC build/autotierfs/autotierfs.o
  CC build/autotierfs/TierEngine/components/mutex.o
  CC build/autotierfs/TierEngine/components/tiering.o
  CC build/autotierfs/TierEngine/components/adhoc.o
  CC build/autotierfs/TierEngine/components/database.o
  CC build/autotierfs/TierEngine/components/base.o
  CC build/autotierfs/TierEngine/components/sleep.o
  CC build/autotierfs/TierEngine/TierEngine.o
  CC build/autotierfs/fuseOps/link.o
In file included from src/impl/autotierfs/fuseOps/link.cpp:24:
src/incl/fuseOps.hpp:265:38: error: use of enum 'fuse_readdir_flags' without previous declaration
  265 |                                 enum fuse_readdir_flags flags);
      |                                      ^~~~~~~~~~~~~~~~~~
make: *** [makefile:51: build/autotierfs/fuseOps/link.o] Error 1
error: builder for '/nix/store/n6jk5wpcdbbs7yzfxccis6fiih2v2fbp-autotier-1.2.0.drv' failed with exit code 2;
       last 10 log lines:
       >   CC build/autotierfs/TierEngine/components/database.o
       >   CC build/autotierfs/TierEngine/components/base.o
       >   CC build/autotierfs/TierEngine/components/sleep.o
       >   CC build/autotierfs/TierEngine/TierEngine.o
       >   CC build/autotierfs/fuseOps/link.o
       > In file included from src/impl/autotierfs/fuseOps/link.cpp:24:
       > src/incl/fuseOps.hpp:265:38: error: use of enum 'fuse_readdir_flags' without previous declaration
       >   265 |                                 enum fuse_readdir_flags flags);
       >       |                                      ^~~~~~~~~~~~~~~~~~
       > make: *** [makefile:51: build/autotierfs/fuseOps/link.o] Error 1
       For full logs, run 'nix log /nix/store/n6jk5wpcdbbs7yzfxccis6fiih2v2fbp-autotier-1.2.0.drv'.

danielbarter · September 28, 2022, 4:20pm

great! The issue is that for fuse3 in nixpkgs, there is an extra folder in the include path.

🍅 nix repl
Welcome to Nix 2.8.1. Type :? for help.

nix-repl> :l /home/danielbarter/nixpkgs
Added 17036 variables.

nix-repl> :b fuse3

This derivation produced the following outputs:
  common -> /nix/store/7jx1z4awqy2hmbvammw6nss080n3z32h-fuse-3.11.0-common
  out -> /nix/store/9vah07iqqwj5qvyikbq237l5argswhws-fuse-3.11.0
[9 copied (18.7 MiB), 3.2 MiB DL]
nix-repl> :q
🍅 cd /nix/store/9vah07iqqwj5qvyikbq237l5argswhws-fuse-3.11.0
🍅 ls
bin  etc  include  lib  sbin  share
🍅 cd ./include/
🍅 ls
fuse3
🍅 cd ./fuse3/
🍅 ls
cuse_lowlevel.h  fuse_common.h  fuse.h  fuse_log.h  fuse_lowlevel.h  fuse_opt.h

so you would need to use #include <fuse3/fuse.h> to have your derivation build without modification. You can fix the derivation by adding the include directory to NIX_CFLAGS_COMPILE. Add the following as an attribute to your derivation:

NIX_CFLAGS_COMPILE = "-I${fuse3}/include/fuse3";

jtojnar · September 28, 2022, 5:52pm

That should not be necessary, the autotier’s build system should call pkg-config fuse3 --cflags and it would receive the correct -I flag.

Looks like they are hardcoding a FHS path, which is just wrong.

jtojnar · September 28, 2022, 5:55pm

This is unrelated but you should not use DESTDIR, see Do not set DESTDIR in makeFlags · Issue #65718 · NixOS/nixpkgs · GitHub.

CafeHaine · September 29, 2022, 10:26am

@danielbarter

so you would need to use #include <fuse3/fuse.h> to have your derivation build without modification. You can fix the derivation by adding the include directory to NIX_CFLAGS_COMPILE . Add the following as an attribute to your derivation:

That was it, thanks a lot !

@jtojnar

That should not be necessary, the autotier’s build system should call pkg-config fuse3 --cflags and it would receive the correct -I flag.

Looks like they are hardcoding a FHS path, which is just wrong.

I’ll try to fix upstream whenever I get this derivation to build!

This is unrelated but you should not use DESTDIR , see Do not set DESTDIR in makeFlags · Issue #65718 · NixOS/nixpkgs · GitHub.

I’ll look into it, thanks a lot!

For now I’m stuck during the embedded build of rocksdb:

rockdb build

[cafehaine@kgenix:~/Documents/Prog/nixpkgs]$ nix-build -A autotier
[…]
  CC build/autotierfs/metadata.o
  CC build/shared/tools.o
  CC build/shared/alert.o
cd src/rocksdb && make USE_RTTI=1 PORTABLE=1 static_lib
make[1]: Entering directory '/build/source/src/rocksdb'
$DEBUG_LEVEL is 0
/nix/store/p7bpdnxqd3i5hwm92mrscf7mvxk66404-bash-5.1-p16/bin/bash: /build/source/src/rocksdb/build_tools/build_detect_platform: /usr/bin/env: bad interpreter: No such file or directory
grep: /build/source/src/rocksdb/make_config.mk: No such file or directory
monitoring/perf_context.cc:23:2: error: #error "No thread-local support. Disable perf context with -DNPERF_CONTEXT."
   23 | #error "No thread-local support. Disable perf context with -DNPERF_CONTEXT."
      |  ^~~~~
monitoring/iostats_context.cc:19:2: error: #error "No thread-local support. Disable iostats context with -DNIOSTATS_CONTEXT."
   19 | #error \
      |  ^~~~~
Makefile:246: make_config.mk: No such file or directory
make[1]: *** No rule to make target 'make_config.mk'.  Stop.
make[1]: Leaving directory '/build/source/src/rocksdb'
make: *** [makefile:54: src/rocksdb/librocksdb.a] Error 2
error: builder for '/nix/store/j3b974w68s4vllsqavzc9kqd91jkvgi6-autotier-1.2.0.drv' failed with exit code 2;
       last 10 log lines:
       > monitoring/perf_context.cc:23:2: error: #error "No thread-local support. Disable perf context with -DNPERF_CONTEXT."
       >    23 | #error "No thread-local support. Disable perf context with -DNPERF_CONTEXT."
       >       |  ^~~~~
       > monitoring/iostats_context.cc:19:2: error: #error "No thread-local support. Disable iostats context with -DNIOSTATS_CONTEXT."
       >    19 | #error \
       >       |  ^~~~~
       > Makefile:246: make_config.mk: No such file or directory
       > make[1]: *** No rule to make target 'make_config.mk'.  Stop.
       > make[1]: Leaving directory '/build/source/src/rocksdb'
       > make: *** [makefile:54: src/rocksdb/librocksdb.a] Error 2
       For full logs, run 'nix log /nix/store/j3b974w68s4vllsqavzc9kqd91jkvgi6-autotier-1.2.0.drv'.

I wonder if I could somehow use the one already packaged in nixpkgs to not have to do it myself

jtojnar · September 29, 2022, 2:25pm

The issue is that the build script hardcodes /usr instead of $(PREFIX)

github.com

45Drives/autotier/blob/5652d837e888f06e2be3125daae0244f7783cd6e/makefile#L106


      
          	-rm -f $(DESTDIR)/usr/bin/$(notdir $(FS_TARGET))
          	-rm -f $(DESTDIR)/usr/bin/$(notdir $(CLI_TARGET))
          
          
tests: view-db
          
          
view-db:
          	mkdir -p dist/tests
          	$(CC) $(CFLAGS) -DBAREBONES_METADATA tests/src/view_db.cpp src/impl/autotierfs/metadata.cpp -Wall -lboost_serialization -lrocksdb -o dist/tests/view_db
          
          
inst-man-pages:
          	mkdir -p $(DESTDIR)/usr/share/man/man8
          	gzip -k doc/man/autotier.8
          	mv doc/man/autotier.8.gz $(DESTDIR)/usr/share/man/man8/
          	ln -snf autotier.8.gz  $(DESTDIR)/usr/share/man/man8/autotierfs.8.gz
          
          
rm-man-pages:
          	-rm -f $(DESTDIR)/usr/share/man/man8/autotier.8.gz
          	-rm -f $(DESTDIR)/usr/share/man/man8/autotierfs.8.gz
          
          
inst-config:
          	mkdir -p $(DESTDIR)/etc

That will need to be fixed.

That just means the file is not executable because /usr/bin/env intentionally does not exist in build sandbox. You can use patchShebangs src/rocksdb/build_tools/build_detect_platform in postPatch to fix the shebang (assuming the program executed by env is on PATH – i.e. in nativeBuildInputs).

That would be preferred.

You would have to remove $(ROCKSDB_STATIC) from target dependencies:

github.com

45Drives/autotier/blob/5652d837e888f06e2be3125daae0244f7783cd6e/makefile#L56


      
          .PRECIOUS: $(TARGET) $(OBJECTS)
          
          
$(FS_OBJECT_FILES) $(CLI_OBJECT_FILES) $(SHARED_OBJECT_FILES): build/%.o : src/impl/%.cpp
          	@mkdir -p $(dir $@)
          	@echo "  CC $@"
          	@$(CC) $(CFLAGS) -c $(patsubst build/%.o, src/impl/%.cpp, $@) -o $@
          
          
$(ROCKSDB_STATIC):
          	cd src/rocksdb && $(MAKE) USE_RTTI=1 PORTABLE=1 static_lib
          
          
$(FS_TARGET): $(FS_OBJECT_FILES) $(SHARED_OBJECT_FILES) $(ROCKSDB_STATIC)
          	@mkdir -p dist/from_source
          	@echo "  LD $@"
          	@$(CC) $(FS_OBJECT_FILES) $(SHARED_OBJECT_FILES) $(ROCKSDB_STATIC) -Wall $(FS_LIBS) -o $@
          
          
$(CLI_TARGET): $(CLI_OBJECT_FILES) $(SHARED_OBJECT_FILES)
          	@mkdir -p dist/from_source
          	@echo "  LD $@"
          	@$(CC) $(CLI_OBJECT_FILES) $(SHARED_OBJECT_FILES) -Wall $(CLI_LIBS) -o $@
          
          
clean: clean-build clean-target clean-rocksdb

And instead use $(shell pkg-config --libs --cflags rocksdb) in these two places:

github.com

45Drives/autotier/blob/5652d837e888f06e2be3125daae0244f7783cd6e/makefile#L59


      
          	@mkdir -p $(dir $@)
          	@echo "  CC $@"
          	@$(CC) $(CFLAGS) -c $(patsubst build/%.o, src/impl/%.cpp, $@) -o $@
          
          
$(ROCKSDB_STATIC):
          	cd src/rocksdb && $(MAKE) USE_RTTI=1 PORTABLE=1 static_lib
          
          
$(FS_TARGET): $(FS_OBJECT_FILES) $(SHARED_OBJECT_FILES) $(ROCKSDB_STATIC)
          	@mkdir -p dist/from_source
          	@echo "  LD $@"
          	@$(CC) $(FS_OBJECT_FILES) $(SHARED_OBJECT_FILES) $(ROCKSDB_STATIC) -Wall $(FS_LIBS) -o $@
          
          
$(CLI_TARGET): $(CLI_OBJECT_FILES) $(SHARED_OBJECT_FILES)
          	@mkdir -p dist/from_source
          	@echo "  LD $@"
          	@$(CC) $(CLI_OBJECT_FILES) $(SHARED_OBJECT_FILES) -Wall $(CLI_LIBS) -o $@
          
          
clean: clean-build clean-target clean-rocksdb
          
          
clean-target:
          	-rm -rf dist/from_source

github.com

45Drives/autotier/blob/5652d837e888f06e2be3125daae0244f7783cd6e/makefile#L103


      
          uninstall: rm-man-pages rm-completion rm-scripts
          	-rm -f $(DESTDIR)$(PREFIX)/$(notdir $(FS_TARGET))
          	-rm -f $(DESTDIR)$(PREFIX)/$(notdir $(CLI_TARGET))
          	-rm -f $(DESTDIR)/usr/bin/$(notdir $(FS_TARGET))
          	-rm -f $(DESTDIR)/usr/bin/$(notdir $(CLI_TARGET))
          
          
tests: view-db
          
          
view-db:
          	mkdir -p dist/tests
          	$(CC) $(CFLAGS) -DBAREBONES_METADATA tests/src/view_db.cpp src/impl/autotierfs/metadata.cpp -Wall -lboost_serialization -lrocksdb -o dist/tests/view_db
          
          
inst-man-pages:
          	mkdir -p $(DESTDIR)/usr/share/man/man8
          	gzip -k doc/man/autotier.8
          	mv doc/man/autotier.8.gz $(DESTDIR)/usr/share/man/man8/
          	ln -snf autotier.8.gz  $(DESTDIR)/usr/share/man/man8/autotierfs.8.gz
          
          
rm-man-pages:
          	-rm -f $(DESTDIR)/usr/share/man/man8/autotier.8.gz
          	-rm -f $(DESTDIR)/usr/share/man/man8/autotierfs.8.gz

Assuming the rocksdb version they are vendoring is not some kind of API-incompatible ancient revision.