Package from multiple github source

1

I want to creates a package from multiple Github repositories. All repos access each other using relative path. I am able to pull the repos using following construct in my derivation:

srcs = [
    (fetchFromGitHub {
      name = xx;
      owner = "ttttt";
      leaveDotGit = true;
      repo = "dfdfdf";
      rev = "";
      sha256 = "";
    })
    (fetchFromGitHub {
      name = rootdir;
      leaveDotGit = true;
      owner = "fdgdf";
      repo = "fgff";
      rev = "";
      sha256 = "";
    })
  ];

But I can not refer other repo from one repo using relative path as they are unpacked in parallel directories with a hash prepended in there name. Is there any way to unpack them to a defined directory without the hash?

2

When using srcs you are supposed to copy your sources into the build dir appropriately during installphase, from there they should be able to reference themselfes appriorpiately.

Another way was to patch them after fetch, which is only possible though if there are no circular references.

Last but not least: You really have to fetch those upstream. Reposirories expecting to be checked out like this really are the hell to work with…

If they need to be like that, then they are effectively a monorepo and should be treated like that.

1 Like
3

I created a mono repo, which includes two submodules. My directory structure looks like:

main-------
          |
          ------Dir1
          |
          ------Dir2

In standard make system, step to configure and build the package is:

> cd Dir1
> SOURCE_X = ../Dir2 make special_config
> make

I tried with following buildPhase override, but it doesn’t work.

buildPhase = ''
   cd Dir1
  SOURCE_X = ../Dir2 make special_config
  make
  '';

I don’t know how to jump to the source directory and issue build command.

4

Using submodules is not “monorepo”. In my opinion it is neither properly seperated projects nor monorepo and should be forbidden as it is forbidden to drive faster than c…

Anyway… I totally do not understand what you do in your current steps nor how it is supposed to be done in practice.

Can you perhaps share the actual project or a minified reproducer?

5 
with import <nixpkgs> {};
let
  rootdir = "buildroot";
  mchipdir = "microchip";
in stdenv.mkDerivation {
  name = "polarfire-sdk";
  src = fetchFromGitHub {
      #name = "microchip-polarfire-sdk";
      owner = "tattvam";
      leaveDotGit = true;
      fetchSubmodules = true;
      repo = "microchip-polarfire-sdk";
      rev = "9a7b26340e45d2877dc3c8b658ec7a9a950cb759";
      sha256 = "sha256-o+5W9J5sYlMtxl+gkFmBKX/JrUTu9hP+TQW5CehQqs4=";
    };

  buildPhase = ''
    BR2_EXTERNAL=../microchip make icicle_defconfig -C buildroot
    make -C buildroot
  '';
  #installPhase = "";
}

In general the package compiles using following steps:

> cd buildroot
> BR2_EXTERNAL=../microchip/ make icicle_defconfig
> make

I want to nixify it using the derivation.

6

Plz ignore unused variables…

7

The tools involve do a lot of unfortunate assumptions…

You will have to do a lot of patching here. In general everything works as expected.

I came a lot further by doing the following changes:

diff -u original.nix default.nix
--- original.nix        2023-02-27 13:27:06.346453083 +0100
+++ default.nix 2023-02-27 13:28:49.209299310 +0100
@@ -14,6 +14,12 @@
       sha256 = "sha256-o+5W9J5sYlMtxl+gkFmBKX/JrUTu9hP+TQW5CehQqs4=";
     };

+  nativeBuildInputs = [which perl unzip];
+
+  postPatch = ''
+    patchShebangs buildroot microchip
+  '';
+
   buildPhase = ''
     BR2_EXTERNAL=../microchip make icicle_defconfig -C buildroot
     make -C buildroot

Now I am stuck as some of the tools wants to find the file executable at /usr/bin/file. This and other hardcoded locations need to be patched out of the build tooling.

And to be honest, if there will be no writes in the BR2_EXTERNAL, then it is not necessary to have the submodules, but then one can use a derivation for the external stuff and just set the env-var appropriately pointing to it.

1 Like