"Stack smashing detected" : packaging woes

Help
1

Hi all

I’m trying to package Macaulay2 on NixOS 23.05.
The build process creates an executable, but any time it is run, it immediately exits with the message

*** stack smashing detected ***: terminated

How can I build Macaulay2 on NixOS and avoid this?
(I have built this repository on other distributions without this problem.)

Development environment:

# shell.nix
{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
  shellHook = 
  ''
    boostLoc=${pkgs.boost}/lib
  '';
  nativeBuildInputs = with pkgs ; [
    autoconf
    automake
    gnumake
    libtool
    bison
    pkg-config
    gfortran
    git
    curl
  ];
  buildInputs = with pkgs ; [
    boehmgc
    tbb
    eigen
    ncurses
    boost
    libzip
    xz
    libxml2
    libffi
    fflas-ffpack
    openblas
    flint
    givaro
    #  gtest # (submodule) You should let M2 compile this: https://github.com/google/googletest/blob/36066cfecf79267bdf46ff82ca6c3b052f8f633c/googletest/docs/faq.md#why-is-it-not-recommended-to-install-a-pre-compiled-copy-of-google-test-for-example-into-usrlocal
    mpir
    gdbm
    readline
    mpfi
    ntl
    glpk
    cddlib # need to pass CPPFLAGS="$CPPFLAGS $(pkg-config --cflags cddlib)/cddlib"  to configure
    gfan 
    singular # Provides factory
    _4ti2
    lrs
    csdp
    nauty
    libatomic_ops
  ];
}

Build process:

git clone 'https://github.com/Macaulay2/M2.git' src
cd src/M2
git checkout release-1.22
make                                      # run autoconf, autoheader
sed -i 's/\/bin\/echo/echo/g' configure   # replace "/bin/echo" with "echo" in the config file

./configure\
  --with-system-gc \              # use the copy of boehmgc on the system
  --prefix=<desired-install-path> \
  --with-boost-libdir=$boostLoc \ # boost library location
   --enable-download \            # allow M2 to download & build more dependencies
  CPPFLAGS="$(pkg-config --cflags cddlib)/cddlib"  # Ensure the system copy of cddlib is found


sed -i 's/\/bin\/echo/echo/g' GNUmakefile
sed -i 's/\/bin\/echo/echo/g' Macaulay2/m2/Makefile.in

make  # Builds the binary, then invokes it to install plugins, triggering the "stack smashing detected" error
# make install, once the errors of make are fixed

A second minor question: how to avoid converting /bin/echo to echo everywhere?
I’ve tried using buildFHSUserEnv, but using it prevents tbb’s headers from being found, even by pkg-config, so the situation is not in net improved.

Thank you for your help!

2

Not sure about your stack smashing issue, but for your echo problem, you can use libredirect to redirect /bin/echo to an appropriate path.

https://github.com/search?q=repo%3ANixOS%2Fnixpkgs+"export+NIX_REDIRECTS"&type=code

1 Like
3

Have a look at macaulay2: init at 1.22 by alois31 · Pull Request #251814 · NixOS/nixpkgs · GitHub. The stack smashing issue is caused by an incompatible BLAS library (openblas is ILP64).

4

Cheers!

By the way, is there anywhere semi-official that this is documented?
The obvious searches aren’t turning up anything for me.

5

I couldn’t find anything. It should probably be added to the Nixpkgs manual, but the existing usages across Nixpkgs should be helpful enough.

6

Oh, looks like you’ve already done all the work for me in packaging this. Thanks!

For future reference, how would I have known BLAS was the root cause here?

7

I reckon strace did the trick

8

I was just lucky to see the assertion in the fflas-ffpack dependency (and I think the incompatibility actually stems from that loading the wrong library).

9

Good to know.
Thanks!

10

I just faced the same error in a nix-shell that uses a pkgs.buildFHSUserEnv from the branch nixos-23.11. After I downgraded my NixOS machine to a 9 day old generation, which came with a Linux downgrade from 6.5.7 to 6.5.5, it worked again and I could enter the nix-shell as always. However, it also could be that something other than Linux caused that change. I didn’t spend too much time to investigate this - for now.

11

I doubt this is related to the issue in this thread, since this was caused by a very specific library incompatibility in Macualay2.

12

are you taling about same ** stack smashing ** error?
i’ve just noticed it started happening for me in untouched fhs environments after one of last system updates.
So i had to update flake locks on those fhs environments, or revert back the system for the stack smashing to go away - I guess some incompatibility happened in one of 23.05 updates…
so much for stable development environments…

13

There is an upstream discussion about this issue: "stack smashing detected" since recent 23.05 upstream change when entering nix-shell with buildFHSUserEnv · Issue #262080 · NixOS/nixpkgs · GitHub

1 Like
14

I’m ashamed that with all googling I did I haven’t stumbled upon that issue…
need to remember to first search specifically in github and discourse, than globally.
At least I’m relieved that smart people noticed that issue and dug into it.

thank you for pointing at that

1 Like