Help with Derivations

Help
1

Hi everyone, I’m trying to create a derivation for an ancient program called QDP++. I am following the instructions on github but I get the error warning: The macro 'AC_TRY_LINK' is obsolete. . Here is my derivation-in-progress:

{
pkgs ? import <nixpkgs> { system = builtins.currentSystem; },
stdenv ? pkgs.stdenv
}:

stdenv.mkDerivation {
  pname = "QDP++";
  version = "1-46-0";
  src = pkgs.fetchFromGitHub {
          owner = "usqcd-software";
          repo = "qdpxx";
          rev = "016182f3c97081204020d6996d3d742583b5d6a9";
          sha256 = "sha256-uxlrSeFcFUYUtnpW8WiimAK9DRF9Ne+sMY23vpVs/zQ=";
        };
   buildInputs = with pkgs; [
    gcc
    autoconf
    automake
  ];
  buildPhase = ''
    autoupdate
    autoconf # trying "./configure.ac" ends up with an error "syntax error near unexpected token `[qdp++],[1.45.0],[edwards@jlab.org]'"
    make
    make install
  '';
  #args = [ ./message.txt ];
}

I think(?) the issue is caused by a backwards compatibility issue with automate. So I’m also trying with autoconf264 and automake115x but this isn’t helping. Wondering if this is a known issue?

The goal is to compile this bad boy, which requires QDP++ as a dependency GitHub - JeffersonLab/chroma: The Chroma Software System for Lattice QCD

2

That’s not an error, that’s a warning.

In any case:

  • remove gcc
  • rename buildInputs to nativeBuildInputs
  • remove the buildPhase

Also, the autotools-related hooks are described in https://nixos.org/manual/nixpkgs/unstable/#setup-hook-autoconf, FYI.

2 Likes
3

Thanks for that. I have modified my derivation:

{
pkgs ? import <nixpkgs> { system = builtins.currentSystem; },
stdenv ? pkgs.stdenv
}:

stdenv.mkDerivation {
  pname = "QDP++";
  version = "1-46-0";
  src = pkgs.fetchFromGitHub {
          owner = "usqcd-software";
          repo = "qdpxx";
          rev = "016182f3c97081204020d6996d3d742583b5d6a9";
          sha256 = "sha256-uxlrSeFcFUYUtnpW8WiimAK9DRF9Ne+sMY23vpVs/zQ=";
        };
   nativeBuildInputs = with pkgs; [
    autoreconfHook
    autoconf
    automake
  ];
}

Now I am getting an error >configure error: Require architecture to be set. Is this something I can set with args, e.g. --march=native? Or is there a more conventional approach?

4

You need to set configureFlags = ["--enable-parallel-arch=${parallel_arch}"]; where parallel_arch is one of scalar, scalarvec, parscalar or parscalarvec. I’m not sure what the difference is between them.

1 Like
5

Thank you. That got the installation going a bit further.

The examples at qdpxx/INSTALL_EXAMPLES at d82326162166849cf208b6d82338a549fff0a194 · usqcd-software/qdpxx · GitHub suggest scalar and parscalar so I have tried those.

For scalar I get > configure: error: libxml2 configuration program xml2-config not found. but I am not sure what should be there for xml2-config.

For parscalar I get:

> configure:  Parscalar build! Checking for QMP
       > checking for qmp-config... no
       > configure: WARNING: QMP configuration program qmp-config not found.
       > configure: WARNING: Set environment variables QMP_CFLAGS QMP_LDFAGS QMP_LIBS
       >         before configure
       > checking if we can compile/link of a simple QMP C++ program... no
       > configure: error: Cannot compile/link a basic QMP C++ program!
       >         Check QMP_CFLAGS, QMP_LDFLAGS, QMP_LIBS.

The INSTALL page also says, among many other things,

Briefly, the shell command ‘./configure && make && make install’
should configure, build, and install this package.

I am wondering if perhaps there is a way I can invoke the ./configure script directly?

6

It appears that qmp is required for parscalar and parscalarvec. I’m not sure what qmp is refering to, however for scalar you can disable libxml2 by adding “–with-libxml2=no” to the configureFlags.

After that it appears that the program fails to compile as it doesn’t include <cstring>

1 Like
7

Thanks - seems you’re right, it doesn’t compile due to the <cstring> issue. Should I use a local src instead then and fix qdi_io.cc? Or is there a more Nix-idiomatic approach?

8

Create a patch and add it to the patches list.
A custom src is also fine too, just seems like overkill here IMO.

9

Thanks again, @waffle8946.

5 or so patched files later and the derivation completes. The files were all just patched to add #include <cstring> or #include <cstdint> except one makefile:

        modified:   include/qdp.h
        modified:   lib/qdp_io.cc
        modified:   lib/qdp_scalar_init.cc
        modified:   lib/qdp_scalar_specific.cc
        modified:   lib/qdp_scalarsite_specific.cc
        modified:   other_libs/Makefile.am

The last file was bugging out on an empty directory, otherlibs/filedb, so I just removed it from the Makefile.

Strangely, the bin folder has only one object in it, and it’s a qdp++-config. Hoping that isn’t because the empty directory is meant to be filled… :slight_smile:

Now I am moving on to trying to compile Chroma. It is also having compilation issues:

> === configuring in other_libs/qdp-lapack (/build/source/other_libs/qdp-lapack)
       > configure: WARNING: no configuration information is in other_libs/qdp-lapack
       > Running phase: buildPhase
       > build flags: SHELL=/nix/store/8vpg72ik2kgxfj05lc56hkqrdrfl8xi9-bash-5.2p37/bin/bash
       > Making all in other_libs
       > make[1]: Entering directory '/build/source/other_libs'
       > Making all in qdp-lapack
       > make[2]: Entering directory '/build/source/other_libs/qdp-lapack'
       > make[2]: *** No rule to make target 'all'.  Stop.
       > make[2]: Leaving directory '/build/source/other_libs/qdp-lapack'
       > make[1]: *** [Makefile:392: all-recursive] Error 1
       > make[1]: Leaving directory '/build/source/other_libs'
       > make: *** [Makefile:528: all-recursive] Error 1

Looking at the relevant makefile, otherlibs/Makefile.am, it contains:

SUBDIRS = qdp-lapack @SSE_DSLASH_DIR@ @CPP_DSLASH_DIR@ @CG_DWF_DIR@ @QOP_MG_DIR@

EXTRA_DIST = cpp_wilson_dslash sse_wilson_dslash cg-dwf

Since qdp-lapack sounds much more relevant, I decided to look into it. Indeed, filedb and qdp lapack are libraries by usqcd.

I’m wondering if there’s an easier way, therefore, to lay out this project. In particular, I’ve got the build input for chroma written in as

  buildInputs = with pkgs; [/nix/store/haaaaaaaash-QDP++-1-46-0];

which seems fragile. Is there a better way to go about this?

10 
buildInputs = [
  (pkgs.callPackage ./qdp.nix { });
];

… assuming the derivation you made earlier is called qdp.nix and in the same directory.

If you want to do it idiomatically and have a proper package set, you’d start with a default.nix somewhat like this:

# default.nix
{
  pkgs ? import <nixpkgs> { }
}: pkgs.lib.makeScope pkgs.newScope (self: {
  qdp = self.callPackage ./qdp.nix { };
  chroma = self.callPackage ./chroma.nix { };
})

Then adjust the derivations to use dependencies from callPackage properly:

# qdp.nix
{
  stdenv,
  fetchFromGitHub,
  autoreconfHook,
  autoconf,
  automake
}:

stdenv.mkDerivation {
  pname = "QDP++";
  version = "1-46-0";
  src = fetchFromGitHub {
    owner = "usqcd-software";
    repo = "qdpxx";
    rev = "016182f3c97081204020d6996d3d742583b5d6a9";
    sha256 = "sha256-uxlrSeFcFUYUtnpW8WiimAK9DRF9Ne+sMY23vpVs/zQ=";
  };
  nativeBuildInputs = [
    autoreconfHook
    # Are these two actually necessary? I thought they're in
    # `stdenv`, or that at least `autoreconfHook` would pull
    # them in implicitly
    autoconf
    automake
  ];
}

# chroma.nix
{
  stdenv,
  # `qdp` is taken from your package set, thanks to `self.callPackage`
  qdp
}:

stdenv.mkDerivation {
  buildInputs = [ qdp ];
}

… now you have a proper package set, which uses the same mechanisms as nixpkgs, including the ability to .override inputs properly. Build your desired package with e.g. nix-build -A chroma.

11

Thanks @TLATER. I was able to reference qdp in a nice way using the first pattern you suggested.

Now I am trying to compile qdp-lapeck, as chroma seems to be having issues with some of the fermion files… Which is definitely not ideal. I’m guessing (hoping) this is related.

I have the following for my qdp-lapeck:

# qdp-lapack
{
pkgs ? import <nixpkgs> { system = builtins.currentSystem; },
stdenv ? pkgs.stdenv
}:

stdenv.mkDerivation {
  pname = "qdp-lapack";
  version = "chroma3-36-1";
  src = pkgs.fetchFromGitHub {
          owner = "JeffersonLab";
          repo = "qdp-lapack";
          rev = "360eec32ff96ff8939b0dafcea243fa56e1d7b3a";
          sha256 = "zc40zFychwKzZ86SJHQoZG9Ccss/N0kBoBtPg7C+Hm8=";
        };

   configureFlags = [
      "--enable-parallel-arch=scalar"
      "--with-libxml2=no"
   ];
   nativeBuildInputs = with pkgs; [
    autoreconfHook
    autoconf
    automake
  ];
  buildInputs = [
  (pkgs.callPackage ../QDP/default.nix { })
];
#   patches = [
#   ./patches/makefiles.patch
#   ];

}

which produces the following error:

 /nix/store/9m68vvhnsq5cpkskphgw84ikl9m6wjwp-coreutils-9.5/bin/mkdir -p '/nix/store/ic3k8c9nwicfgkp87vjnllbd72yzvfz7-qdp-lapack-chroma3-36-1/include'
 /nix/store/9m68vvhnsq5cpkskphgw84ikl9m6wjwp-coreutils-9.5/bin/install -c -m 644  qdp-lapack.h fortran_lapack.h qdp-lapack_IncrEigpcg.h qdp-lapack_Complex.h qdp-lapack_numerical.h qdp-lapack_eigpcg.h qdp-lapack_numerical_private.h qdp-lapack_fortran_lapack.h qdp-lapack_ortho.h qdp-lapack.h qdp-lapack_restart_X.h '/nix/store/ic3k8c9nwicfgkp87vjnllbd72yzvfz7-qdp-lapack-chroma3-36-1/include/.'
/nix/store/9m68vvhnsq5cpkskphgw84ikl9m6wjwp-coreutils-9.5/bin/install: will not overwrite just-created '/nix/store/ic3k8c9nwicfgkp87vjnllbd72yzvfz7-qdp-lapack-chroma3-36-1/include/./qdp-lapack.h' with 'qdp-lapack.h'
make[2]: *** [Makefile:337: install-nobase_includeHEADERS] Error 1
make[2]: Leaving directory '/build/source/include'
make[1]: *** [Makefile:478: install-am] Error 2
make[1]: Leaving directory '/build/source/include'
make: *** [Makefile:359: install-recursive] Error 1

This mention of not being able to overwrite might be to do with the Nix store, so I obviously can’t change that. What am I to do? Is this going to lead to me patching the makefile for coreutils? That feels like it could quickly spiral.