Workspace style multi-repo code workflows with nix

Help
1

Can one have a “workspace” like workflow with nix, flakes and multiple repositories, including nixpkgs? Here’s what I mean, what I have now, and it seems to mostly work, but I’m afraid, and there are parts where it doesn’t. I’m curious to hear how other people are approaching setups like this.

With “workspace-like”, here’s what I’d like to have:

  1. Allow individual direnvs to work in individual directories (subrepositories) of the project.
  2. Avoid needing to push every development version to github or some similar public URL for testing to make nix happy.
  3. Avoid needing to push to github fixed paths to my home directory. Currently I’m failing this. :frowning_face: My nix_observe/flake.nix points to path:/home/sliedes/proj/observe/nixpkgs.
  4. Being able to work in an IDE like vscode neatly so that I add the three directories, nix_observe, nixpkgs and top in a vscode workspace as folders.

Currently I have a directory structure like

workspace/.git
workspace/nix_observe/.direnv
workspace/nix_observe/.git
workspace/nix_observe/flake.nix
workspace/nixpkgs
workspace/nixpkgs/.git
workspace/top/.direnv
workspace/top/flake.nix

(Context for the curious: nix-observe is a transparent wrapper for executing programs and observing the execution. Very much WIP and experimental, but I want to bring visibility to builds. It is intended to wrap command executions, like nix-observe -- gcc -c foo.c -o foo — also for ld, make, ninja, … . I can discuss that too if someone is curious, but we might want to do that in a different topic. I don’t make any claims of it being suitable for anything, yet.)

top/flake.nix is intended to implement whatever is needed to make nixpkgs use the wrapper around whatever I want to wrap. nixpkgs is there because I plan to edit it too. And the whole setup is structured as a directory tree in order to avoid necessary round trips to github, which is a pain point I’ve had when working with nix as a nix beginner.

Non-goals:

While I ended up using git submodules to make nix happy, tracking precise version dependencies between the three git repositories while developing is at this point a non-goal.

Questions:

  1. Can I somehow reasonably have nixpkgs.url in nix_observe/flake.nix point to some generally reasonable place (like github nixpkgs), yet persistently get direnv to use ../nixpkgs for it when I’m under nix_observe/? It just seems wrong to have something in github pointing to a path in my computer.
  2. Is there some silly pitfall?
  3. Can this be done without git submodules? Or are they just the right tool for this?

In my top/flake.nix, I do this:

{
  inputs = {
    self.submodules = true;
    nixpkgs.url = ../nixpkgs;
    nix-observe = {
      url = ../nix_observe;
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
  ...
}

While this overrides nix_observe.nixpkgs to be what I want for the top flake, it doesn’t help direnv when inside nix_observe/…

1 Like
2

Ok, I found the pitfall. Every comment change to top/flake.nix results in a new copy of nixpkgs in the store. :stuck_out_tongue: