Any default config for reproducible configurations?

Is there any default template to easily create NixOS configs?

In my opinion the true NixOS config must follow next principles:

  • Use pinned (in git) version of nixpgs in some way. With git submodule or fetch.json file.
  • Ability to install packages from multiple versions of nixpkgs in same environment. For example install all packages from stable branch but the telegram install from nixpkgs master because of new features.
  • No use of nix-env. All packages must be installed from config
  • Support of multiple devices. One repository must be able to contain configuration code for multiple devices. For example user may have a home server, a desktop and a laptop. All devices use same configuration repository with small (or big) differences. For example the server must not have X11 installed, but have DHCP and DNS servers installed and configured. The desktop has X11, dev tools and games. The laptop is like the desktop but with no games. The laptop and the desktop share same set of users with same passwords.
  • Easy overlays. With script or template to create a new one.
  • Immutable users by default.
  • Some way to safely manage secrets (immutable users passwords, hardware configuration) by default.
  • Maybe some scripts to automatically update all pinned versions of nixpkgs.
  • Maybe use the home-manager by default? I’ve never used it, but dudes say it is cool replacement for the users.users.userName.packages with ability to manage configurations in the home dir.

So the question is: is there any template for NixOS configs following above principles?

Like, I fork it on GH, then enter my own packages set, my secrets, my overlays, my tools configurations and other stuff, and that’s it. I have ready to use fully featured easily supportable the true NixOS config.

6 Likes

can you elaborate on this? Like, a command like nixos-rebuild edit but for overlays?

Well, at least there must be some default empty overlay in the config template to let the user just to type in their derivations.

Great questions!

  • Use pinned (in git) version of nixpgs in some way. With git submodule or fetch.json file.

I recommend using git submodule on your own fork of nixpkgs. I never use this folder for development so the tree is always clean.

  • Ability to install packages from multiple versions of nixpkgs in same environment. For example install all packages from stable branch but the telegram install from nixpkgs master because of new features.

Use import ( fetchFromGitHub {[...]} ) to grab from specific branch of nixpkgs. For more automation, check out niv

  • No use of nix-env. All packages must be installed from config

This seems trivial?

  • Support of multiple devices. One repository must be able to contain configuration code for multiple devices. For example user may have a home server, a desktop and a laptop. All devices use same configuration repository with small (or big) differences. For example the server must not have X11 installed, but have DHCP and DNS servers installed and configured. The desktop has X11, dev tools and games. The laptop is like the desktop but with no games. The laptop and the desktop share same set of users with same passwords.

Break up your config into different files (“modules”) and from configuration.nix, import relevant modules with

imports =
    [ # Include the results of the hardware scan.
      /etc/nixos/hardware-configuration.nix
      ../../profiles/zfs.nix
      ../../profiles/common.nix
      ../../profiles/kde.nix
  ]
  • Easy overlays. With script or template to create a new one.

Symlink from your git repo. I do ln -s /Computer/home /home/tyler/.config/nixpkgs where /Computer is my repo. Now add overlays to e.g. /Computer/home/overlays/my-overlay.nix.

  • Immutable users by default.

users.mutableUsers = false;

  • Some way to safely manage secrets (immutable users passwords, hardware configuration) by default.

git-crypt

  • Maybe some scripts to automatically update all pinned versions of nixpkgs .

See my nixos-update below.

  • Maybe use the home-manager by default? I’ve never used it, but dudes say it is cool replacement for the users.users.userName.packages with ability to manage configurations in the home dir.

100% do this! I mostly use /Computer/home/home.nix rather than configuration.nix.

So the question is: is there any template for NixOS configs following above principles?

Additional helper scripts:

{ pkgs, lib, ... }:
with pkgs;
let
  shebang-cd = ''
    #!${stdenv.shell}
    set -e # terminate on exit(1)
    cd /Computer
  '';
  nix-script-common = shebang-cd + ''
    git submodule init
    git pull --no-edit --recurse-submodules
  '';
  nix-quick = writeScriptBin "nix-quick" (shebang-cd + ''
    home-manager -I nixpkgs=/Computer/nixpkgs switch --show-trace
  '');
  nix-sync = writeScriptBin "nix-sync" (nix-script-common + ''
    home-manager -I nixpkgs=/Computer/nixpkgs switch --show-trace
    git add home/* bin/* shell/* packages/* nixops/* org/* README.md
    git commit -m "nix-sync succeeded on $(date)"
    git push
  '');
  nixos-update = writeScriptBin "nixos-update" (nix-script-common + ''
    git submodule update --remote --init --recursive
    home-manager -I nixpkgs=/Computer/nixpkgs switch --show-trace
    sudo nixos-rebuild -I nixpkgs=/Computer/nixpkgs boot
    nix-channel --update
    sudo nix-channel --update
    git add .
    git commit -m "nixos-update succeeded on $(date)"
    git push
  '');
    
  nixos-sync = writeScriptBin "nixos-sync" (nix-script-common + ''
    home-manager -I nixpkgs=/Computer/nixpkgs switch --show-trace
    sudo nixos-rebuild -I nixpkgs=/Computer/nixpkgs boot
    git add .
    git commit -m "nixos-sync succeeded on $(date)"
    git push
  '');

  nixos-switch = writeScriptBin "nixos-switch" (nix-script-common + ''
    home-manager -I nixpkgs=/Computer/nixpkgs switch --show-trace
    sudo nixos-rebuild -I nixpkgs=/Computer/nixpkgs switch
    git add .
    git commit -m nixos-switch succeeded on "$(date)"
    git push
  '');
in
{
  home.packages = [
    nix-diff
    nixops
    nix-prefetch-github
    nix-prefetch-git
    nix-sync
    nix-quick
    nixos-update
    nixos-sync
    nixos-switch
    nixpkgs-pytools
  ];
}

Hope this helps!

2 Likes

That is exactly what I have for myself in GitHub - binarin/nixos-config: NixOS configurations for all my machines (except for secrets and auto-updates). Your post even made me to update a README on it =).

But the idea to have something like this as a pre-made template is really great. I spent a lot of time digging through the source code to make it work. And I assume a lot of other people with the same requirements also did so.

2 Likes

edit

I have started an announcements page for further discussion if anyone is interested.


Well actually I’ve been working on just such a project using the new flakes feature. There is a damn solid skeleton in place that makes getting up and running simple. The README.md could tell you more about the directory heirarchy, but you should be able to do everything you asked much simpler now. In addition, every custom package, overlay and module declared are automatically bundled into the flake output for use in external configurations. Oh, and I flakified home-manager and wired it up to be included in every configuration by default!

If you start a nix-shell in the base directory, it will automatically set up and pull in an experimental version of nix that supports flakes so you can get up and running quickly. I even have a simple command line script rebuild in the shell which will take the place of nixos-rebuild for now.

It should be as simple as a:

# from project root directory
nixos-generate-config --show-hardware-config > ./hosts/<preffered-hostname>.nix

# flakes are vcs based so a brand new file must at least be staged for it to be contained in the flake
git add ./hosts/<preffered-hostname>.nix

rebuild <preferred-hostname> test

in the nix-shell to deploy a bare bones config to your current hardware. From there simply drop your current configuration in a subdirectory of profiles and import it from the hosts/<preferred-hostname>.nix. It’s great for getting up and running imediately, then you could start porting your configuration to match the ideals of the directory structure to keep concerns neatly separated.

In fact I was considering writing an RFC to include what I have as part of the NixOS organization on GitHub. I feel having a standardized and sane directory structure for configurations would go a long way for ease of sharing and contribution. I could actually really use some feedback, or any ideas that could possibly make it better!

6 Likes

@nrdxp very cool! I’m aware of flakes but not that educated on the topic. Could you expand on the advantage of your approach vs what @binarin & I described? I think I get the gist but don’t understand the why.

Say your friend and you are using this repository, each with your own unique nix epxpressions. By simply importing your friends flake from flake.nix as an input, you can have access to all of the packages, modules, overlays, and even entire system configurations your friend has defined!

As in, overlays wouldn’t clash?

General advantages include everything automatically delivered by flakes, ie:

  • deterministic evaluation.
  • Easily pinning nixpkgs, as well as using multiple nixpkgs versions.
  • easily exporting packages, modules, or whole system configurations to other flakes.
  • overriding or importing other flakes to extend configurations trivially

Advantages specific to my repo:

  • Super easy to get up and running
  • Every file in hosts/<file>.nix is automatically added as a valid NixOS configuration available in flake outputs, without having to wire it up yourself.
  • Super modular, allowing multiple system declarations. And clean separation of various concerns.
  • Resuse of different profiles between different machines, and mix and matching profiles and sub-profiles to easily build a NixOS configuration from existing code is as trivial as an imports statement.
  • Secrets automatically encrypted using git-crypt
  • All flake outputs wired up automatically to so you can focus on writing interesting nix expressions rather than boilerplate.
  • If adoption catches on, having a standardized directory structure would make switching contexts, say from one job to the next a lot simpler and faster.

answer to overlays question

Well I’ve wired up the flake to automatically export any packages defined in the pkgs directory as an overlay which can then be imported into another flake. This should be trivially combinable with your own overlays.

Concrete example.

{
  # flake.nix
  inputs.<friends-flake>.url = "<flake-uri>";

  outputs = args@{ self, nixpkgs, <friends-flake> }: {
    ...
  };
} 

# hosts/default.nix -- basis for all configurations
# it is the only file with direct access to all flake inputs
...
  global = {
    ...
    nixpkgs.overlays = self.overlays ++ args.<friends-flake>.overlays; 
    ...
  };
...  

The workflow for adding a package to the overlay is essentially exactly the same as adding a package to nixpkgs itself. This is intential to provide a kind of staging area for new package development.

The only difference, is instead of adding a top level package definition to all-packages.nix as in nixpkgs, you add the package definition to pkgs/default.nix in the exact same way. The rest is taken care of automatically by flake.nix.

Since flakes are still not merged into master, there are still some limitations that will be resolved once it is. For example, the flakes pr repo is a bit out of date, so for now I am pointing nixpkgs to a fork that is just the flakes pr rebased onto a more recent version of master.

Once it actually gets merged you can trivially specify multiple versions of nixpkgs as inputs by simply given them unique names and specifying their flake uri directly, e.g. “nixpkgs/release-19.09” or whatever the finalized uri semantics end up looking like.

1 Like

Thanks, very interesting! I can see the advantages in terms of hermetic sharing of config.

One concern I have is about having multiple pinned nixpks is in regards to space usage. I have a lot of compute-heavy packages installed, and already my nix store hovers around 80-100GB, and that’s with a single nixpkgs version pinned. Even a single generation is probably 60+ GB.

Nix is already space-hungry, and it seems like flakes will exacerbate this?

Well not necessarily. My nix install takes up about 7.5G, which is actually less that when I used to use ArchLinux, which was typically around 9G. Just to pull in a nixpkgs as an input only takes up about 18Mb. Then add in whatever you might install from it.

I hate to state the obvious, but just in case… Have you run nix-store --optimize to free up space and do you run the nix-collect-garbage -d frequently or at all to clean up old generations? Just asking, because 60G for a single generation seems pretty extreme!

Yes, I do both. I just looked more into it using ncdu /nix, and it seems like CUDA is almost singularly responsible…as to not derail this thread I made a new post here: Duplicate CUDA derivations with minor differences. Let me know if you have any ideas!

just of curiosity, whats your

du -sch $(nix-store -qR /run/current-system) | grep total

,

du -sch $(nix-store -qR ~/.nix-profile) | grep total

and

nix-store -qR ~/.nix-profile | grep glibc | wc -l

?

❯ du -sch $(nix-store -qR /run/current-system) | grep total
11G	total
/nix                                                                                                                                                                        0.72 tyler@lensman
❯ du -sch $(nix-store -qR ~/.nix-profile) | grep total
20G	total
/nix                                                                                                                                                                        1.49 tyler@lensman
❯ nix-store -qR ~/.nix-profile | grep glibc | wc -l
12

Note that all my cudatoolkit dependencies are installed via home-manager. I use shells as well but I believe they are all garbage collected (haven’t added a root).

1 Like

I’ve just tried to build your config template and the nix-shell does not work.

% nix-shell                                                                                                                         ~/work/nixflk
error: invalid character '/' in name 'opt/nix.conf'
(use '--show-trace' to show detailed location information)

% nix-shell --show-trace                                                                                                            ~/work/nixflk
error: while evaluating the attribute 'NIX_CONF_DIR' of the derivation 'nix-shell' at /etc/nixos/nixpkgs/pkgs/build-support/mkshell/default.nix:32:3:
invalid character '/' in name 'opt/nix.conf'

I haven’t experience this issue at all, perhaps a link to the repo you are running nix-shell from would reveal more insight.

How are the secrets used? Do they end up in the store?

Depends on how you use them. The secrets directory is setup to automatically encrypt everything inside of it. How you use it is up to you. For example, I have my user password hashes saved there, and import them from the users directory and they never end up in the store.

However, my ssh keys are also used from the secrets directory, and because of the way they are used, they do end up in the store. Check out my default user configuration for a few examples of how this is done. Of course it would be perferable to not have this world readable, but upstream has not yet mainlined a solution for this, but there are some proposals open on the issue tracker, such as this one.

1 Like