How to include FFI Cabal Flags?

Help
1

Hello,

I’m trying to get hasktorch compiled in nix using haskell.nix. Currently I’m encountering an error during one of the sub libraries of hasktorch called libtorch-ffi. I believe the error is caused by the inability (missing foreign-library support · Issue #465 · NixOS/cabal2nix · GitHub) for cabal2nix to pickup the foreign libraries flags in libtorch-ffi’s .cabal file. This results in the following error during compilation:

[ 43 of 105] Compiling Torch.Internal.Managed.Native.Native9 ( src/Torch/Internal/Managed/Native/Native9.hs, dist/build/Torch/Internal/Managed/Native/Native9.p_o )
[ 44 of 105] Compiling Torch.Internal.Managed.Native ( src/Torch/Internal/Managed/Native.hs, dist/build/Torch/Internal/Managed/Native.p_o )
[ 45 of 105] Compiling Torch.Internal.Unmanaged.Optim ( src/Torch/Internal/Unmanaged/Optim.hs, dist/build/Torch/Internal/Unmanaged/Optim.p_o )
<command-line>: error: expected identifier before numeric constant
<command-line>: error: expected ‘}’ before numeric constant
In file included from /nix/store/75jl927al5mblqd0l42i0103955fswj4-python3.10-torch-2.0.0.0-dev/include/torch/csrc/jit/api/function_impl.h:5,
                 from /nix/store/75jl927al5mblqd0l42i0103955fswj4-python3.10-torch-2.0.0.0-dev/include/torch/csrc/jit/api/method.h:7,
                 from /nix/store/75jl927al5mblqd0l42i0103955fswj4-python3.10-torch-2.0.0.0-dev/include/torch/csrc/jit/api/object.h:6,
                 from /nix/store/75jl927al5mblqd0l42i0103955fswj4-python3.10-torch-2.0.0.0-dev/include/torch/csrc/jit/api/module.h:4,
                 from /nix/store/75jl927al5mblqd0l42i0103955fswj4-python3.10-torch-2.0.0.0-dev/include/torch/csrc/api/include/torch/serialize/input-archive.h:6,
                 from /nix/store/75jl927al5mblqd0l42i0103955fswj4-python3.10-torch-2.0.0.0-dev/include/torch/csrc/api/include/torch/serialize/archive.h:3,
                 from /nix/store/75jl927al5mblqd0l42i0103955fswj4-python3.10-torch-2.0.0.0-dev/include/torch/csrc/api/include/torch/nn/pimpl.h:5,
                 from /nix/store/75jl927al5mblqd0l42i0103955fswj4-python3.10-torch-2.0.0.0-dev/include/torch/csrc/api/include/torch/optim/adagrad.h:3,
                 from /nix/store/75jl927al5mblqd0l42i0103955fswj4-python3.10-torch-2.0.0.0-dev/include/torch/csrc/api/include/torch/optim.h:3,

                 from /build/ghc3054_0/ghc_275.cpp:8:0: error: 

/nix/store/75jl927al5mblqd0l42i0103955fswj4-python3.10-torch-2.0.0.0-dev/include/torch/csrc/jit/runtime/graph_executor.h:19:28: error:
     note: to match this ‘{’
       19 | enum ExecutorExecutionMode {
          |                            ^
   |
19 | enum ExecutorExecutionMode {
   |                            ^
<command-line>: error: expected unqualified-id before numeric constant

Here is the section in libtorch-ffi’s cabal file.

 cxx-sources:          csrc/hasktorch_finializer.cpp
 c-sources:            csrc/hasktorch_dump.c
 install-includes:     csrc/hasktorch_finializer.h
                      ,csrc/hasktorch_dump.h
 include-dirs:         csr

I’ve tried adding csrc to libtorch-ffi’s project settings in cabalProject'

          packages.libtorch-ffi = {                                                                                                                                      
            configureFlags = [                                                                                                                                           
              "--extra-include-dirs=${lib.getDev torch}/include/torch/csrc/api/include"                                                                                  
              "--extra-include-dirs=${../libtorch-ffi/csrc}"                                                                                                             
            ];                                                                                                                                                           
            flags = {                                                                                                                                                    
              cuda = cudaSupport;                                                                                                                                        
              rocm = false;                                                                                                                                              
              gcc = !cudaSupport && stdenv.hostPlatform.isDarwin;                                                                                                        
            };                                                                                                                                                           
          };

But unfortunately the error remains. Any ideas on how I can fix this?

2

haskell.nix is completely independent of cabal2nix.

3

Oh yeah you’re right. Sorry about that. I was thinking of haskell-flake. I don’t have much direct evidence then of if haskell.nix is using those flags then. I don’t see any reference to them in the log file, only the configure flags. The derivation doesn’t yield much insight either.

4

Doesn’t hasktorch have a full Nix infrastructure for building with haskell.nix?

If this isn’t working for you, maybe making an issue on their repo would be an easier way to get help.

There also seems to be a closed issue that might be close to what you’re asking about:

https://github.com/hasktorch/hasktorch/issues/671

1 Like
5

I’m trying to fix the infrastructure right now. Currently there is a shellFor included with hasktorch. The shellFor works for using cabal build hasktorch on CPU but not GPU. I would like to avoid using this method and discover exactly why hasktorch doesn’t build inside Nix.

At the moment there is only one maintainer who I’ve been speaking to on the project’s discord server. Recently he has suspended working on nix to focus his attention on the source code.

Thanks for linking this. I’ll see if the forks mentioned there can help me.

1 Like
6

I figured out the issue. Enabling profiling with haskell.nix causes the libtorch-ffi to be built twice and during the second time it errors. Disabling profiling fixes the issue allowing hasktorch to be built.

1 Like