How do you map `pip install git+https://github.com/colcon/colcon-common-extensions.git` in a nix flake?

I am currently trying to work my way into the nix-ros-overlay. However, what I want to do with it is of a more specialised nature. I used to work with ros2 and a docker container, but now I’m not so happy with the solution. That’s why I wanted to do more with flakes, since I also use a nixos. That’s my current solution:


{
  inputs = {
    nix-ros-overlay.url = "github:lopsided98/nix-ros-overlay/develop";
    nixpkgs-unstable.url = "github:nixos/nixpkgs?ref=nixos-unstable";
    nixpkgs.follows = "nix-ros-overlay/nixpkgs"; # IMPORTANT!!!
  };
  outputs =
    {
      self,
      nix-ros-overlay,
      nixpkgs,
      nixpkgs-unstable,
    }:
    nix-ros-overlay.inputs.flake-utils.lib.eachDefaultSystem (
      system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ nix-ros-overlay.overlays.default ];
          config.allowBroken = true;
        };
        pkgs-unstable= import nixpkgs-unstable {
          inherit system;
          config.allowBroken = true;
          };
      in 
      {
        devShells.default = pkgs.mkShell {
          name = "nix_ros";
          packages = [
            # ... other non-ROS packages
            pkgs.colcon
            pkgs.cargo-ament-build
            # unstable packages
            pkgs-unstable.rustc
            pkgs-unstable.bash-completion
            pkgs-unstable.curl
            pkgs-unstable.wget
            pkgs-unstable.git
            pkgs-unstable.clang
            pkgs-unstable.libclang
            pkgs-unstable.vcstool
            pkgs-unstable.uv
            (pkgs-unstable.python312.withPackages (ps: [
              ps.argcomplete
              ps.polars
              ps.numpy
              ps.altair
              ps.matplotlib
              ps.pip
            ]))
            (
              with pkgs.rosPackages.jazzy;
              buildEnv {
                paths = [
                  ros-base
                  ros-core
                  robot-localization
                  rosbag2
                  rsl
                ];
              }
            )
          ];
          shellHook = ''
            echo "Environment working right now"
            '';
        };
      }
    );
  nixConfig = {
    extra-substituters = [ "https://ros.cachix.org" ];
    extra-trusted-public-keys = [ "ros.cachix.org-1:dSyZxI8geDCJrwgvCOHDoAfOm5sV1wCPjBkKL+38Rvo=" ];
  };
}

How get I something like pip install git+https://github.com/colcon/colcon-common-extensions.git into the flake?

You would need to package that python package. You can take the src either via a fetcher like fetchurl or fetchFromGithub or via a flake input.

You’re right. I’ll stay with the docker container.

That’s the thing I’ve come up with:

{ lib
, buildPythonPackage
, fetchFromGitHub
}:

buildPythonPackage rec {
  pname = "colcon-common-extensions";
  version = "0.3.0";  # Update this to the latest version

  src = fetchFromGitHub {
    owner = "colcon";
    repo = "colcon-common-extensions";
    rev = "version-${version}";  # Update this to match the specific git tag/commit
    sha256 = ""; # Add the hash after first attempt
  };

  # Enable tests if available
  doCheck = true;

  meta = with lib; {
    description = "Common extensions for the colcon build tool";
    homepage = "https://github.com/colcon/colcon-common-extensions";
    license = licenses.asl20;  # Verify the correct license
    maintainers = [ ];
  };
}

This is the default.nix file.
While calling `nix-build -E ‘with import { }; python3Packages.callPackage ./default.nix { }’
it produced the following:

warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
these 2 derivations will be built:
  /nix/store/l9g3qjl8ch4qbvm2f9zy0acv2bkpgni2-source.drv
  /nix/store/981frxz5fsin8xqsf1hn7p4asds4qsz5-python3.12-colcon-common-extensions-0.3.0.drv
building '/nix/store/l9g3qjl8ch4qbvm2f9zy0acv2bkpgni2-source.drv'...

trying https://github.com/colcon/colcon-common-extensions/archive/version-0.3.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404
error: cannot download source from any mirror
error: builder for '/nix/store/l9g3qjl8ch4qbvm2f9zy0acv2bkpgni2-source.drv' failed with exit code 1;
       last 8 log lines:
       >
       > trying https://github.com/colcon/colcon-common-extensions/archive/version-0.3.0.tar.gz
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > curl: (22) The requested URL returned error: 404
       > error: cannot download source from any mirror
       For full logs, run 'nix log /nix/store/l9g3qjl8ch4qbvm2f9zy0acv2bkpgni2-source.drv'.
error: 1 dependencies of derivation '/nix/store/981frxz5fsin8xqsf1hn7p4asds4qsz5-python3.12-colcon-common-extensions-0.3.0.drv' failed to build

How to read that?

It cannot find the file to download because there is no tag with the name version-... , so you need to use the correct format:

  src = fetchFromGitHub {
    owner = "colcon";
    repo = "colcon-common-extensions";
-   rev = "version-${version}";  # Update this to match the specific git tag/commit
+   rev = version;  # Update this to match the specific git tag/commit
    sha256 = ""; # Add the hash after first attempt
  };

It is also available on PyPI (link) so you can use fetchPypi instead:

  src = fetchPypi {
    inherit pname version;
    hash = lib.fakeHash; # Replace it with the hash from the error.
  };
1 Like

Ok, I’ve got the following:

{ lib
, buildPythonPackage
, fetchFromGitHub
}:

buildPythonPackage rec {
  pname = "colcon-common-extensions";
  version = "0.3.0";  # Update this to the latest version

  src = fetchFromGitHub {
    owner = "colcon";
    repo = "colcon-common-extensions";
    rev = version;  # Update this to match the specific git tag/commit
    sha256 = "WgE+33zJb44O7wv1+7vgRnR+Mq7UKn7FFDS6MP8sAjQ="; # Add the hash after first attempt
  };

  # Enable tests if available
  doCheck = true;

  meta = with lib; {
    description = "Common extensions for the colcon build tool";
    homepage = "https://github.com/colcon/colcon-common-extensions";
    license = licenses.asl20;  # Verify the correct license
    maintainers = [ ];
  };
}

And a folder with the result. What to do with that? Can I somehow add this to my nix store or should I submit it to the global nix store? Can I somehow build this against a specific python version?

in let…in under your pkgs definitions do colcon-common-extensions = pkgs.python3Packages.callPackage […] (or use pkgs-unstable if you want)

and you should be able to use this package below in packages = [] without pkgs. in the beginning

edit: this won’t work with a meta package, at least this one. you can however package stuff from colcon-common-extensions’s install_requires this way if it’s missing from nixpkgs

Why do you want to do that? I think we have all colcon-common-extension in the nix-ros-overlay. See here. Just add pkgs.colcon to the flake and you should be done.

2 Likes