You cant use the /var/lib in the installing phase because the derivation dont have the necessary permissions.
I couldnt find any documentation about using the directories outside the nix store like /var, despise that topic should be, i think, abundant, as lot of packages needs stateful directories.
There is this topic, about system.activationScripts and modules but is little information and i coudnt understand what a shell script that runs everytime you boot has to do with installing /var.
So… is there some example of how that works or a place to find more info on that?
/var is the domain of NixOS modules, not packages. All the activation script does is make sure the correct root files and directories exist so the correct systemd services and tmpfiles rules (declared in the module) can create those directories on-demand. This is in contrast to FHS distros, where /var files are mixed in with the rest of the package, whereas those are explicitly split into immutable derivation state and mutable /var filesystem state in NixOS.
(I think this should make sense, let me know if not!)
Activation scripts are generally discouraged, as they run at a critical point in the boot process, serially, and failure is catastrophic. If you just need to copy some files into /var/lib or create a directory or something, this is definitely the situation in which to use tmpfiles (the name is not representative; it’s used for persistent files all the time)
That make sense, but if /var is domain of nixOS modules, does that mean that nix packages installed in foreign distros with nix package manager dont use /var?
And is there documentation explaining how to use modules to package things?
FWIW I think this is a pretty big overstatement. There are way more packages in nixpkgs than there are package integrations in nixos modules, because most software doesn’t actually need anything like that and works just fine on other distros.
Anyway, @gabr I think we need to know more about your actual use case to provide more useful guidance for non-NixOS distros.
I don’t think so nixpkgs packages really only work well on NixOS, tried the foreign distro thing for nearly a year, it’s just a mess. Besides, I’m making the assumption that anything that needs access to /var is probably some server-focused app, which wants some config, and just about every other distro will provide some default config, which NixOS may also do without a module due to environment.pathsToLink, but a package installed imperatively on a foreign distro will never benefit from that.
I mean I think the existence of darwin as a usable platform in nixpkgs is proof that it’s fine. There’s nix-darwin but it doesn’t have modules for most things and also plenty of people don’t use it.
pathsToLink doesn’t actually do anything most of the time. It just adds things to /run/current-system/sw/, but this isn’t generally useful. Some NixOS things consider it, like where to look for manpages or .desktop applications, but for the most part it’s not really relevant. It certainly doesn’t result in default configs being installed or discovered by most software. The fact is, most software just generates its default config when it doesn’t already exist.
Okay, well, I’m not going to encourage people to use obviously broken-by-design setups like nixpkgs packages on a foreign distro. I don’t know anything about darwin, I do know enough about linux to see how broken it is. I’ve had enough people complaining about glibc mismatches and whatnot to know that it’s basically the mixing-stable-with-unstable problem but magnified, because nixpkgs and the foreign distro’s packaging don’t talk to each other.
And I’ve seen some (probably badly-written, but whoever said all software was good?) software segfault or panic if a config isn’t explicitly provided. But really my point is, on a foreign distro you often have to manually configure software in a way that you don’t have to on NixOS. Things don’t necessarily “just work” on NixOS either without configuration, there’s just a higher chance of this being possible (compared to a foreign distro) since there are some packages that (if they do generate a default config as you describe, and come with a systemd service file) all you’d likely need is to put them in systemd.packages (where environment.pathsToLinkis relevant AIUI).
OP’s confusion is likely that a package from a foreign distro’s native packaging solution includes slapping arbitrary files on the filesystem (services, /etc config, perhaps even statefully creating folders in /var), with nix that’s never the case as it always sits in the nix store, so even systemd services won’t be present from an imperative install.
I should also note that the reason for some server-focused packages not having modules is sometimes, no one got around to writing such a module (or reveiwing the existing PR for said module). Though that should be obvious, I guess I have to state it.
My use case is not non-nixOS specific, i just want to package a program that needs a place to put /var. Im just curious about how /var installation seems to be only possible in NixOS, I thought that the majority of packages use /var, /etc, how can so many packages exist in nix on foreign distro? do all of them not just /var nor /etc?
edit:
my nix expression is the following, do i need to write a module to install the database of urbackup in /var?
with import <nixpkgs> {};
stdenv.mkDerivation rec {
pname = "urbackup-server";
version = "2.5.20";
src = fetchFromGitHub {
owner = "uroni";
repo = "urbackup_backend";
rev = "refs/tags/${version}";
hash = "sha256-brP7nYikbIvQP/K+qSKyd8eHWUIK9Ai/4qP09ptr+lE=";
};
buildInputs = [
cryptopp
curlWithGnuTls
autoconf
automake
libtool
pkg-config
# gnum4
# zlib
# zstd
];
configurePhase = ''
runHook preConfigure
bash switch_build.sh server
autoreconf --install
sh configure --prefix=$out \
--with-crypto-prefix=${ builtins.storePath cryptopp.dev }
runHook postConfigure
'';
# --localstatedir="/var/lib"
doCheck = true;
enableParallelBuilding = true;
meta = with lib; {
description = "Easy to setup Open Source client/server backup system";
longDescription = "An easy to setup Open Source client/server backup system, that through a combination of image and file backups accomplishes both data safety and a fast restoration time";
homepage = "https://www.urbackup.org/index.html";
license = licenses.agpl3Plus;
platforms = platforms.linux;
};
}
A lot of things do, but it’s when they are run as system services. Nixpkgs provides a lot of packages, but they don’t all require being started as a systemd unit. Again, that’s the domain of modules.
If this software is looking for a state directory and is hardcoded to /var, but doesn’t require a systemd service, but also could work in $XDG_DATA_HOME or $XDG_STATE_HOME (see also) when run by a user, that’s where you’d write a wrapper and wouldn’t need a NixOS module.
Interesting. I guess urbackup, as a backup software, should need systemd services, so i problably need to write a module for it.
but its still not clear to me if i could install a backup software, for example, in Debian with nix package manager (as a nix package, not debian package), as it would need systemd units and that is not provided by nix, only in nixOS with nix modules.
edit: urbackup is not hardcoded to /var, but i guess it would be preferable to install in a system directory, not home directory.
I think you’ve found one of the largest impedance mismatches in the Nix ecosystem here. It would be great if this worked, but I haven’t found any solutions short of running NixOS as a VM or LXC container (though, I think I’ve seen system-manager which I have not tried but does look promising). I run into the same problem with nix-on-droid taken to a more extreme level - it gives me a shell in an environment that uses Nix, and home-manager obviously works, but there are no system services.
Then it’s possible to package it in such a way that it works on both NixOS and fhs-based distros. All you need to do is flip on the option to put it in another directory in the NixOS module to have it function on NixOS, and you leave the default /var directory in the normal package.
If it’s a systemd service, you can alternatively use e.g. systemd’s StateDirectory setting - this would allow you to completely skip hardcoding any path, systemd would figure out at runtime which path to write to, and provide it for you, so the package would work with no changes both on NixOS and fhs-based distros.
It generally is pretty easy to make these things work, just requires actually knowing general Linux packaging/configuration conventions and the thing you’re packaging well enough to put this together. And then a fair bit of effort to actually make something coherent.
That said, yeah, not everything is that simple, see e.g. the mess that is graphics drivers @waffle8946 refers to.
If you don’t explain a bit more closely why you want to touch /var and for what, it’s hard to help you here. I took a glance at the upstream project, it has too little obvious documentation to understand what you’re referring too.
Its an issue, but its fine. At least i can now say that to people who think NixOS is not necessary because there is the package manager that “works everywhere and does basically the same thing”.
All you need to do is flip on the option to put it in another directory in the NixOS module to have it function on NixOS, and you leave the default /var directory in the normal package
At which directory would NixOS install /var things? i thought it installed in /var too, only that to do that it needs modules.
that StateDirectory config is put into the module, right?
If you don’t explain a bit more closely why you want to touch /var and for what, it’s hard to help you here. I took a glance at the upstream project, it has too little obvious documentation to understand what you’re referring too.
Urbackup has a database file where it constantly writes, and that is normally put into /var/lib. In nix i cant write to /var/lib, at least not in the package definition.
If i understand correctly i need to write a package module to instruct systemd to make the /var/lib directories and to the backup service to work.
As it needs systemd it cant run in non-NixOS distros.
My question would be: if the package dont need systemd, E.G. a package that only needs to read /etc/packagename. Would it be possible to be installable in non-NixOS distros.
Note that im referring to /etc and /var but it dont need to be that specifc directory, it only needs to be a writable directory acessible by the package binaries, and that is the same for all users.
“Installing” in /var sounds a bit paradoxical as /var is for state.
If the program itself is somehow not copying the starter db on first run, then you could use some module using e.g. systemd-tmpfiles to copy it there. Systemd-tmpfiles management via nix on foreign distros would work via something system-manager (which I’ve used before, it works for /etc and systemd unit management, nothing more.)
I think you misunderstood - StateDirectory is a setting in the systemd service file which is set via system.services.<name>.serviceConfig in a NixOS module or via a system-manager module. But this is ultimately systemd functionality.
This is silly. State dirs like this should be created at runtime. But we didn’t write the code. The way to handle this is to set it to /var (or /var/lib for modern fhs compliance, I think), and then to patch out the code that creates the directory (like this package).
You then write a wrapper with makeWrapper to create the directory at runtime instead. This makes the package work on NixOS and non-NixOS hosts, since the wrapper would create the path at runtime, which would permit it to write to /var even on NixOS.
For bonus points, you can patch the code to do state dir allocation at runtime, and ideally listen to the systemd-defined $STATE_DIRECTORY environment variable to determine the path, then upstream that. This will make the software just work with systemd, and allow any system to set up state dirs as needed - much better!
For the record, all of this is only necessary for projects that hard-code paths, which is quite frankly just bad practice. This won’t work universally on fhs-based distros either, since some prefer to use /usr/var. I personally wouldn’t use this software on account of this, if you can’t get the basics right…
nixpkgs packages really only work well on NixOS, tried the foreign distro thing for nearly a year, it’s just a mess. Besides, I’m making the assumption that anything that needs access to