The flex package is not installed,causing the gengtype-lex.c file is missing under the GCC construction

when I got gcc source from gcc-mirror(https://github.com/gcc-mirror/gcc),and build it with nix,it complained missing gengtype-lex.c。I found current standard environment don’t include flex package,

would it be better if nixpkgs include flex,when building gcc package?

thanks!

flex seems to be available:

nix shell nixpkgs#nix-index -c nix-locate bin/flex | grep -v \(
lispPackages.flexi-streams.out                      424 x /nix/store/jfz1hamrzdzn9nys4x84nrh8yrf2p38d-lisp-flexi-streams-20200925-git/bin/flexi-streams-lisp-launcher.sh
flexget.out                                       1,221 x /nix/store/ns3ma9jfvg65fvpb5mnlylqxh4j1zp8g-FlexGet-3.1.127/bin/flexget
flexget.out                                       1,230 x /nix/store/ns3ma9jfvg65fvpb5mnlylqxh4j1zp8g-FlexGet-3.1.127/bin/flexget-headless
flex_2_5_35.out                                 347,768 x /nix/store/hb0fg8vlpyi2hmxgcwx2y6inaklkg272-flex-2.5.35/bin/flex
flexcpp.out                                     609,728 x /nix/store/22sglqpdiljkgijz1gz88kjaciljad4a-flexc++-2.05.00/bin/flexc++
flex.out                                        358,968 x /nix/store/53f3m3p9hcdjab2jhrssci0y1qqah26b-flex-2.6.4/bin/flex
flex.out                                              0 s /nix/store/53f3m3p9hcdjab2jhrssci0y1qqah26b-flex-2.6.4/bin/flex++
ejabberd.out                                     39,628 r /nix/store/x2q1a6pyg2f86nazgnsf1z8jifcqnbsc-ejabberd-20.12/lib/xmpp-1.5.1/ebin/flex_offline.beam

gcc releases don’t require flex and thus derivations don’t pull it in as a dependency. If you want to build a gcc snapshot or gcc from git tree you will need to add flex as an explicit dependency into your derivation.

yes,nixpkgs can install flex

you are right,I will try,thanks!


Do you know why rebuilding gcc9 reported this error,and stop building?

It’s too little detail to say definitely. Usually nix shows a hint to run nix log /path/to/drv to get more detailed log. But in this case my guess is that SIGSEGV is a sign of out-of-memory condition. dmesg should show if OOM happened and what was resource usage of processes at OOM time.

If you don’t have enough RAM you might want to try lower parallelism with --cores.

self: pkgs:
with pkgs;
{
  gcc9 = lowPrio (wrapCC (gcc9.cc.overrideAtrrs ( oldAttrs : rec  {
           src = [ ./gcc-9.4.0.tar.gz ];
           patches=...
           buildInputs = (oldAttrs.buildInputs or [] ) ++ [ flex ];
  })));
}

the overlay file for gcc9 is like above. Once I build gcc9 with this overlay,it‘s been stuck in the building of bootstrap-stage0-stdenv-linux.drv,it show the error:


And machine memory is enough,110G avaliable。
Last but not least,this overlay file works for gcc7,8,10,and 11,except for gcc9.
Have you ever sucessfully rebuilt gcc9 package ?

Aha, 110G is more than enough.

I build gcc9 (as 9.4.0) from staging locally a few times a week without any problems. I don’t apply custom patches to it though. I suggest looking at the build log to get better idea where SIGSEGV happens…

And is the overlay I wrote similar to you? the patches attr can ignore。And the output of nix log /nix/store/xxxx-bootstrap-stage0-stdenv-linux.drv shows nothing…
I also posted the detailed issue on github,haha Failed to build gcc9.4 due to signal 11 (Segmentation fault) · Issue #148391 · NixOS/nixpkgs · GitHub

Which nix version you are using? I personally patch gcc right in nixpkgs/master checkout and don’t use an overlay. If you don’t see a log perhaps the overlay definition is wrong. Does it take a while to fail for you?

If I use your overlay I get the error:

$ cat ~/.config/nixpkgs/overlays/a.nix
self: pkgs:
with pkgs;
{
  gcc9 = lowPrio (wrapCC (gcc9.cc.overrideAtrrs ( oldAttrs : rec  {
           src = [ ./gcc-9.4.0.tar.xz ];
           patches=[];
           buildInputs = (oldAttrs.buildInputs or [] ) ++ [ flex ];
  })));
}
$ nix build -f. gcc9
error: attribute 'overrideAtrrs' missing

       at /home/slyfox/.config/nixpkgs/overlays/a.nix:4:27:

            3| {
            4|   gcc9 = lowPrio (wrapCC (gcc9.cc.overrideAtrrs ( oldAttrs : rec  {
             |                           ^
            5|            src = [ ./gcc-9.4.0.tar.xz ];
(use '--show-trace' to show detailed location information)

I’ll spend some time to understand why it fails and will post working overlay if i succeed.

try both 2.3.16 and 2.4pre-rc1,It takes a long time to be failure.And the overlay don’t have semantic error,and the buildInputs can remove,because the build is stop before gcc compilation(flex raises problem you stated in github).

# gcc_overlay.nix
self: pkgs:
with pkgs;
{
  gcc9 = lowPrio (wrapCC (gcc9.cc.overrideAttrs ( oldAttrs : rec  {
           version = "9.4.0";
           src = [ ./gcc-9.4.0.tar.gz ];
  })));
}

# gcc.nix
let
  overlay = import ./gcc_overlay.nix;
  pkgs = import <nixpkgs> { overlays = [ overlay ]; };
in pkgs.gcc9

just run nix-build gcc.nix,I don’t use nixOS。

After fixing 2 typos (overrideAttrs with 2 t but only 1 r and adding import in gcc.nix) it failed during patchPhase, as all patches are skipped, if I understand the log correctly:

If I then also add patches = []; to the override it starts building. And as far as I can tell it skipped the bootstrapping…

Lets see how long it takes, then I’ll report back.

1 Like

Works on my machine:

./result/bin/gcc --version
gcc (GCC) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I built from nixos/nixpkgs commit 6daa4a5c045d40e6eae60a3b6e427e8700f1c07f (nixpkgs-unstable)

1 Like

Oh great catch on the typo @NobbZ! I confirm the following tweaks allow me to build gcc9:

--- a/gcc.nix
+++ b/gcc.nix
@@ -2 +2 @@ let
-  overlay = ./gcc_overlay.nix;
+  overlay = import ./gcc_overlay.nix;
diff --git a/gcc_overlay.nix b/gcc_overlay.nix
index ada8026..72a34e7 100644
--- a/gcc_overlay.nix
+++ b/gcc_overlay.nix
@@ -4 +4 @@ with pkgs;
-  gcc9 = lowPrio (wrapCC (gcc9.cc.overrideAtrrs ( oldAttrs : rec  {
+  gcc9 = lowPrio (wrapCC (gcc9.cc.overrideAttrs ( oldAttrs : rec  {
@@ -6 +6,2 @@ with pkgs;
-           src = [ ./gcc-9.4.0.tar.gz ];
+           src = [ ./gcc-9.4.0.tar.xz ];
+           patches = [];
1 Like

sorry, I manually wrote the code on this post with some typos,but the code on my server don’t have typo。Then I will try to add the patches=[],and build again,thanks !

I am so sorry for misleading you,and thanks for providing me the detailed diff file,I’ll try it.

1 Like

I noticed that building gcc8 don’t have this step building '/nix/store/z3gcsyzg8fjb29qxb1zdn7psqzngx082-bootstrap-stage0-stdenv-linux.drv,but gcc9 happenned. And I am still failed in this, not reaching patchPhase.
When you built gcc9, was the bootstrap-stage0-stdenv-linux.drv ever built ?I don’t know why it built bootstrap-stage0-binutils-wrapper-.drv,one of its inputDrvs is just bootstrap-stage0-stdenv-linux.drv.I think, when I built gcc9,it trigerred the bootstapping of binutils and glibc and others.

From https://raw.githubusercontent.com/NixOS/nixpkgs/master/pkgs/top-level/all-packages.nix:

  inherit (let
      num =
        if (with stdenv.targetPlatform; isVc4 || libc == "relibc") then 6
        else if (stdenv.targetPlatform.isAarch64 && stdenv.isDarwin) then 11
        else if stdenv.targetPlatform.isAarch64 then 9
        else 10;
      numS = toString num;
    in {
      gcc = pkgs.${"gcc${numS}"};
      gccFun = callPackage (../development/compilers/gcc + "/${numS}");
    }) gcc gccFun;
  gcc-unwrapped = gcc.cc;

On x86_64 gcc9 is not a bootstrap compiler and just a mere package. You don’t need to rebuild much to get it. arm64 is the only target where it’s the case.

If you need to use flex in a bootstrap compiler you might also need to use flex: use fetchurl instead of fetchpatch to avoid extra depends by trofi · Pull Request #147678 · NixOS/nixpkgs · GitHub to avoid circular dependency of stdenv on itself.

I forgot to say the build platfrom is aarch64-linux, so when I overlay the gcc9,it will build with bootstrap.You may be make experiment on x86,right?