NixFS: Access flakes Through Filesystem Paths

Announcements
1

Hey everyone!
Introducing NixFS, a FUSE that allows you to access any Nix derivation simply by accessing a corresponding path. NixFS aims to simplify the process of exploring the Nix store and accessing derivations without having to explicitly build them first.

How NixFS Works

NixFS uses a simple directory structure:

/nixfs
└── flake
    ├── b64
    └── str

When you access a path like /nixfs/flake/str/nixpkgs#hello, NixFS will automatically trigger a build of nixpkgs#hello . The path will then behave as a symlink to the store path of the build result.

For flakes with / in their URL, access via /nixfs/flake/str will make parsing the flake URI too complicated. You can instead access these flakes using the b64 directory, like so: /nixfs/flake/b64/<base64 encoded flake URL> .

Why?

NixFS was created to address challenges faced when working with distributed compute frameworks like Apache Spark, which often expect you to provide static paths to Python binaries. These frameworks also sometimes propagate absolute paths for binaries from the application driver to executors that may be running on different hosts.

By providing a filesystem-based interface to access Nix derivations, NixFS allows users to seamlessly integrate Nix packages and their dependencies into distributed computing workflows. This eliminates the need for complex workarounds or manual path adjustments when dealing with these frameworks.

Usage

To mount the NixFS filesystem, run:

$ nix run github:illustris/nixfs -- [--debug] [<fuse mount options>] /mount/path
$ /mount/path/flake/str/nixpkgs#hello/bin/hello
Hello, world!

NixOS Module Integration

NixFS can also be integrated as a module in your NixOS configuration. To do this, add the following to your flake.nix :

{
	inputs.nixfs.url = "github:illustris/nixfs";
	
	outputs = {nixpkgs, nixfs, ...}: {
		nixosConfigurations.my_machine = {
			imports = [ nixfs.nixosModules.nixfs ];
			services.nixfs.enable = true;
		};
	};
}

This will enable the NixFS service on your NixOS machine, automatically mounting the filesystem on startup.

Contribute to NixFS

Feel free to submit issues or pull requests on the GitHub repository.

30 Likes
2

Interesting! Do you have an example on how to use this to run Spark?

1 Like
3

This is what using custom python flakes with dependencies would look like without nixfs:

  1. pyWrapper: a script that builds and invokes the pythonWithPackages
#!/usr/bin/env bash

mkdir -p /tmp/nix-cache
export HOME=/tmp/nix-cache

# spark home contain pyspark and py4j src zips
# need to do this, otherwise the pyspark version from the source spark-submit is used
PYTHON_PATH=$SPARK_HOME/python/lib:$SPARK_HOME/python/:$PYTHON_PATH

/run/current-system/sw/bin/nix --extra-experimental-features nix-command --extra-experimental-features flakes run --quiet "$(readlink -f ./environment/)"#myenv -- "$@"
  1. flake.nix
{
	inputs = {
		nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
	};

	outputs = { self, nixpkgs, ... }: {
		packages = nixpkgs.lib.genAttrs [ "x86_64-linux" "x86_64-darwin" ] (sys: let
			pkgs = nixpkgs.legacyPackages.${sys};
		in {
			myenv = pkgs.python3.withPackages (p: with p; [
				lightgbm
				...
			]);
		});
	};
}

  1. zip and upload the files to HDFS

  2. submit the application to the cluster

export PYSPARK_PYTHON=./environment/pyWrapper
export PYSPARK_DRIVER_PYTHON=$PYSPARK_PYTHON
spark-submit --archives <path to zip in hdfs> #environment app.py

With NixFS, you could skip the bundling of the environment entirely and do something like:

export PYSPARK_PYTHON=/nixfs/flake/str/mach-nix#gen.python.lightgbm/bin/python
export PYSPARK_DRIVER_PYTHON=$PYSPARK_PYTHON
spark-submit app.py

If you have a more complicated environment, you can point to a git repo containing the flake definition

/nixfs/flake/b64/$(echo -n github:illustris/nixfs | base64)/bin/nixfs
2 Likes
4

I know this is total overkill for what this is built for, but I look forward to trying this out to test my home-manager config without NixOS-rebuilding.

1 Like