`make-shell`, a modular replacement for `mkShell`

Turn THIS:

pkgs.mkShell {
  packages = [pkgs.curl pkgs.git pkgs.jq pkgs.wget];
};

into THIS!

pkgs.make-shell {
  packages = [pkgs.curl pkgs.git pkgs.jq pkgs.wget];
};

:sweat_smile: Alright, I admit that this project wonā€™t help many simple use cases, but changing mkShell to accept a module as its argument is helping me manage a herd of devShells, using imports to manage common features, and to define new options.

I wrote a full rationale for the project in a WHY.md document, which might tell you if you could get any use from it, but if you prefer to see that by example, hereā€™s a ā€œShell Moduleā€ which adds and authenticates a configurable Google Cloud CLI package:

{
  lib,
  pkgs,
  config,
  ...
}: {
  options.gcloud.enable = lib.mkEnableOption "install google cloud sdk";
  options.gcloud.extraComponents =
    lib.mkOption
    {
      description = "IDs of Google Cloud SDK components to install (list them with `gcloud components list`)";
      default = [];
      example = ["gke-gcloud-auth-plugin"];
      type = lib.types.listOf lib.types.nonEmptyStr;
    };
  config = lib.mkIf config.gcloud.enable {
    packages = [
      (
        if config.gcloud.extraComponents == []
        then pkgs.google-cloud-sdk
        else
          pkgs.google-cloud-sdk.withExtraComponents
          (builtins.map (c: pkgs.google-cloud-sdk.components.${c})
            config.gcloud.extraComponents)
      )
    ]; 
    shellHook = ''
        gcloudAccount=$(gcloud auth list --filter=status:ACTIVE --format="value(account)")
        if [ -z ''${gcloudAccount:+set} ]; then
          echo "Google Cloud SDK is not authorized"
          read -rp "In the browser tab about to open, authenticate to your Google Cloud account. (press enter to continue)"
          gcloud auth login
        fi
        if ! gcloud auth application-default print-access-token&>/dev/null; then
          read -rp "You don't have any default application credentials for Google Cloud. In the browser tab about to open, authorize the use of your Google Cloud account. (press enter to continue)"
          gcloud auth application-default login
        fi
    '';
  };
}

ā€¦and hereā€™s a simple option which uses that ā€œgcloudā€ module to set a developer up to authenticate with clusters running in google cloudā€™s kubernetes engine:

{
  lib,
  pkgs,
  config,
  ...
}: {
    options.k8s.enable = lib.mkEnableOption "kubernetes tools";
    config = lib.mkIf config.k8s.enable {
        packages = [pkgs.kubectl pkgs.minikube];
        gcloud.enable = true;
        gcloud.extraComponents = ["gke-gcloud-auth-plugin"];
    };
};

Commit those files to a flake and then you could write a consuming flake like this:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
    make-shell.url = "github:nicknovitski/make-shell";
    my-org-nix.url = "github:myOrg/nix-config"
  };

  outputs = {
    self,
    nixpkgs,
    flake-utils,
    make-shell,
    my-org-nix,
  }:
    flake-utils.lib.eachDefaultSystem (system: let
      pkgs = import nixpkgs {
        inherit system;
        config = {};
        overlays = [make-shell.overlays.default];
      };
    in {
      devShells.gcloud-function = pkgs.make-shell {
        imports = [my-org-nix.shellModules.default];
        gcloud.enable = true;
      };
     devShells.kubernetes = pkgs.make-shell {
       imports = [my-org-nix.shellModules.default];
       k8s.enable = true;
     };
    });
}

It also has a flake.parts flake module, if you like those. I hope at least a few people find it useful! :smile:

17 Likes

The argument to each of these fns is identical. As-is, I donā€™t see the point of pkgs.make-shell at all.

Could you come up with an example that actually demonstrates the virtues pkgs.make-shell is supposed to have?

1 Like

Please read the rest of the post

3 Likes

I did, and yet I wrote that post. Whatā€™s that tell ya?

Well, the rest of the post is an example of using it in a way that shows off what itā€™s useful for. Itā€™s mkShell but modular, because the input is a NixOS module, that can include other modules and configuration/options.

2 Likes

:+1: Thank you very much for sharing make-shell. It is exactly the lean alternative to existing options Iā€™d been hoping someone would write.

1 Like

Haha! Iā€™m not bothered if make-shell seems useless to you; as my joking first example was meant to show, in many situations, it is useless! :smile:

Also I do struggle to explain where it isnā€™t useless in a small number of words. In the projectā€™s WHY, I try to describe the situations where I think people could get the most benefit from it, and highlight all of the contrasting situations where I think they wouldnā€™t. I wrote that document because I know there are many, many users of nix, and even of mkShell who have no strong reason to use make-shell, and I want them to be able to recognize that as soon as possible.

I linked the WHY document instead of copying it to my original post because itā€™s long, but I can summarize the primary experience that made me want to write make-shell: it was working across 10+ flakes in one organization, all with at least one devShell, in all of which developers expected a few common dependencies to be present, but which some developers didnā€™t want to install globally.

Hereā€™s how to achieve that with make-shell:

# my-org.nix/flake.nix
https://flake.parts/options/make-shell
outputs: _ {
  shellModules.default = { pkgs, ...}: { packages = [pkgs.jq pkgs.yq pkgs.fd]; };
}
# 
# our-python-project/flake.nix
inputs.make-shell.url = "github:nicknovitski/make-shell";
inputs.my-org-nix = "path:../my-org.nix";
outputs = {
    nixpkgs,
    flake-utils,
    make-shell,
    my-org-nix,
    ...
  }:
    flake-utils.lib.eachDefaultSystem (system: let
      pkgs = import nixpkgs {
        inherit system;
        config = {};
        overlays = [make-shell.overlays.default];
      };
    in {
  devShells.default = pkgs.make-shell {
    imports = [my-org-nix.shellModules.default];
    packages = [ pkgs.python3 ]; # option definitions here are merged with the ones in the imported modules
  };
}

Is that a clearer use-case?

3 Likes

Thank you so much for saying this, thatā€™s exactly the use I hope people have from it. Itā€™s true that make-shell is just a much, much worse version of devenv and devshell. Itā€™s much less user-friendly, it has no CLI, no config file standard, almost no ā€œfeaturesā€ at all! But if youā€™re repeating yourself in writing shell definitions for your flakes, you could use it to write as many of those missing features which you want and have time for. And maybe we can move towards open source collections of re-usable Shell Modules which could be re-used by those superior projects? I intend to write one of those next, to see if itā€™s a good idea or not.

Oh, and the flake moduleā€™s documentation has been published on flake.parts now.

2 Likes

I like things that are much worse. Have you considered offering a variant that makes a ā€œbareā€ shell without any of the usual stdenv stuff? Getting a direnv setup that doesnā€™t set 500 environment variables or override the macOS shell utilities with GNU coreutils is most of why I use devshell.

3 Likes

I have written modules which make a little progress towards that, although I havenā€™t decided whether or where to publish them.

One of make-shellā€™s options is function, which is pkgs.mkShell by default, but can be replaced with any function which supports the same arguments. And one actual ā€œfeatureā€ that make-shell has over mkShell is that the env option can contain attributes whose value is null, which causes the variable of that attributeā€™s name to be unset in the shell.

So a simple module which used both of those features to slim things down a bit could look like this:

{pkgs,...}: {
  function = pkgs.mkShellNoCC;
  env = { # credit to https://github.com/cachix/devenv/blob/7b3ed618571f0d14655b05f7b1c6ef600904383a/src/modules/top-level.nix#L79
    HOST_PATH = null;
    NIX_BUILD_CORES = null;
    __structuredAttrs = null;
    buildInputs = null;
    buildPhase = null;
    builder = null;
    depsBuildBuild = null;
    depsBuildBuildPropagated = null;
    depsBuildTarget = null;
    depsBuildTargetPropagated = null;
    depsHostHost = null;
    depsHostHostPropagated = null;
    depsTargetTarget = null;
    depsTargetTargetPropagated = null;
    dontAddDisableDepTrack = null;
    doCheck = null;
    doInstallCheck = null;
    nativeBuildInputs = null;
    out = null;
    outputs = null;
    patches = null;
    phases = null;
    preferLocalBuild = null;
    propagatedBuildInputs = null;
    propagatedNativeBuildInputs = null;
    shell = null;
    shellHook = null;
    stdenv = null;
    strictDeps = null;
  };
}

But Iā€™m sure this could be improved a great deal, especially for function.

1 Like

Yeah, I think starting from scratch rather than trying to trim mkShell down to size makes more sense. devshell has mkNakedShell, but itā€™s not publiclyā€exposed (at least in the flake).

1 Like

Iā€™ve been subscribed to this PR for a while, it may show a good way forward, whether or not itā€™s right for nixpkgs: mkShellMinimal: Create an ultra minimal nix-shell

1 Like

I always thought my PR for a minimal shell then adding layers (I guess it could be modules) could be a viable alternative.

You could build up the default mkShell back from it.

I wish I landed that PR you linked :sob:

2 Likes

I thought it might also be helpful to have a fully worked shell module as an example, so I made this: GitHub - nicknovitski/javascript-nix: `make-shell` nix modules for javascript development.

Actually, I know it will be helpfulā€¦to me! But I hope it will be helpful to other people too, whether to see whatā€™s possible with make-shell, or as the start of configuration for a project or two.

e: I already hope to extend it to have a version-selection capability in the vein of the very impressive nixpkgs-python.

Oh, interesting. So it wouldnā€™t be impossible to maintain the mkShell interface while slimming down the implementation to something like your minimal shellā€¦that seems promising!

It definitely is, thank you for that explanation :slight_smile:

What is the differential for this compared to cachix devenv?

1 Like

As I said above, but I donā€™t mind saying again, the difference is itā€™s much worse! Itā€™s a function you can call in your flakes which takes a module and returns a shell derivation, and thatā€™s it. devenv is a command line tool which does not require you to work with flakes, has many great subcommands like search and up, includes I-donā€™t-know-how-many option declarations contributed and maintained by I-donā€™t-know-how-many people (including myself), and is supported by a great company. make-shell has zero high-level option declarations, and I wasnā€™t planning on adding any, except in other flakes like javascript-nix.

If youā€™re evaluating options for adopting nix for a project, my advice is you shouldnā€™t even consider make-shell. Maybe thatā€™s something I should emphasize in the why fileā€¦

I just released v1.2.0, which has a big not-breaking change: the deprecation of the function option and the addition of the stdenv option.

I think makes a little more sense; function always felt strange to me.

let
  thisEmitsAWarning = make-shell ({pkgs, ...}: { function = pkgs.mkShellNoCC; });
  useThisInstead = make-shell ({pkgs, ...}: { stdenv = pkgs.stdenvNoCC; });
in
  thisEmitsAWarning == useThisInstead
3 Likes

Iā€™ve released another shell module, GitHub - nicknovitski/gcloud-nix: A shell module for adding the Google Cloud SDK.

The Google Cloud SDK includes the gcloud command line tool, which has a components subcommand for installing a variety of other tools and packages. Some of those tools have separate packages in nixpkgs, but all of the components are also packaged as components in nixpkgs, as an interesting interface for the package google-cloud-sdk:

pkgs.google-cloud-sdk.withExtraComponents [
  pkgs.google-cloud-sdk.components.kubectl
  pkgs.google-cloud-sdk.components.gke-gcloud-auth-plugin
  pkgs.google-cloud-sdk.components.skaffold
]

gcloud-nix lets me make this terser, and merges component definitions, so I can do something like this:

# gke.nix
{pkgs, ...}: {
  gcloud.enable = true;
  gcloud.extra-components = [ "kubectl" "gke-gcloud-auth-plugin" "skaffold" ];
}

# For most projects:
pkgs.make-shell {
  imports = [./gke.nix];
  # project-specific options
}

# For a project which also uses pubsub:
pkgs.make-shell ({pkgs, ...}: {
  imports = [./gke.nix];
  gcloud.extra-components = [ "pubsub-emulator" ]; # concatenated cleanly with gke.nix!
  packages = [ pkgs.openjdk ]; # necessary for the emulator to run
  shellHook = "gcloud beta emulators pubsub env-init"; # set environment variables so pubsub applications run in this shell will target the emulator
})

Relatedly, Iā€™m also considering upstreaming an option to make-shell which Iā€™ve found use for in a few projects: interactiveShellHook. It basically does this:

config.shellHook = lib.mkAfter ''
    if [[ -n "''${_nix_direnv_force_reload:-}" || $- == *i* ]]; then
      ${config.interactiveShellHook}
    fi
'';

The conditional is true when someone is interactively running nix-shell or nix-develop, or nix-direnv-reload from nix-direnvā€™s manual mode (GitHub - nix-community/nix-direnv: A fast, persistent use_nix/use_flake implementation for direnv [maintainer=@Mic92 / @bbenne10]). Otherwise it is false.

mkAfter means that this will be added to the end of shellHook, which made sense to me: conditionally evaluated code coming after code which is always evaluated.

It can be used for things like this:

pkgs.make-shell {
  env.AWS_PROFILE = "project-profile";
  packages = [pkgs.awscli2];
  interactiveShellHook = ''
    if ! aws configure list; the
      echo "Set your AWS access key id and secret access key! (Other values can be blank)"
      aws configure --profile "$AWS_PROFILE"
    fi
  '';
}

pkgs.make-shell {
  gcloud.enable = true;
  gcloud.extra-components = ["gke-gcloud-auth-plugin" "kubectl"];
  interactiveShellHook = "gcloud container clusters get-credentials myClusterName";
}
3 Likes