Hi,
I’m struggling to create a comfortable sandbox environment for ad-hoc running untrusted code (let’s assume that all pip/node packages is potential malware). The scenario works like this:
- clone a repo with command line tool
- create a flake from template
- run nix shell
- install dependencies using package manger (
pip install -r requirements, etc.)
- run the code inside bubblewrap jail so: home is not polluted with packages - they are dropped when I exit the shell and python/node cmd can only see CWD and eventually has some other restrictions bubblewrap is capable of
There is nix-bwrapper or more recent jail.nix but they work as simple wrappers. I can force python/node to be inside bubblewrap but then I have to provide dependencies as nix packages. My aim is to be able to quickly and safely run a python tool and get rid of it, not to spend time untangling issues with package compatibility.
I tried to create a virtualenv that is bubblewrapped like this:
{
description = "A very basic flake";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
jail-nix.url = "sourcehut:~alexdavid/jail.nix";
};
outputs =
{
self,
nixpkgs,
jail-nix,
...
}@inputs:
let
system = "x86_64-linux";
pkgs = import nixpkgs { inherit system; };
jail = jail-nix.lib.init pkgs;
shell_packages = [
(jail "python3"
(pkgs.python3.withPackages (py: [
py.requests
py.pip
py.virtualenv
]))
[
jail.combinators.network
jail.combinators.mount-cwd
]
)
pkgs.openssl
];
in
{
devShells.${system} = {
default = pkgs.mkShell {
packages = shell_packages;
};
};
packages.${system}.default = pkgs.symlinkJoin {
name = "env";
paths = shell_packages;
};
};
}
This approach does not work though - python created by virtualenv is not bubblewrapped and this is quite error prone.
So I decided to look for a simpler, less error prone option: let’s just spawn a whole shell that is bubblewrapped and drop whatever additional binaries I need inside. Give the shell some writable FHS dirs in tmpfs and let’s roll. I haven’t implemented a POC yet, because my first issue is to find a clean way to start such shell. Can I override the shell binary that gets launched during nix shell or nix develop? My other idea is just to bind my shell dotfiles as readonly into the jail and use nix run mechanism to spawn a shell.
Perhaps there is a project that solves this issue already?
I came up with something that roughly works by using nix run:
{
description = "A very basic jail flake";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
jail-nix.url = "sourcehut:~alexdavid/jail.nix";
};
outputs =
{
self,
nixpkgs,
jail-nix,
...
}@inputs:
let
user = "myuser";
system = "x86_64-linux";
pkgs = import nixpkgs { inherit system; };
jail = jail-nix.lib.init pkgs;
python3-env = pkgs.python3.withPackages (py: [
py.requests
py.pip
py.virtualenv
]);
pentest-env = pkgs.symlinkJoin {
name = "pentest-env";
paths = [
python3-env
pkgs.zsh
pkgs.openssl
pkgs.coreutils
pkgs.gcc # often needed for pip compiling
pkgs.fzf
];
};
jailed-shell = jail "zsh" "${pentest-env}/bin/zsh" [
jail.combinators.network
jail.combinators.mount-cwd
(jail.combinators.add-path "${pentest-env}/bin")
(jail.combinators.ro-bind "/home/${user}/.zsh" "/home/${user}/.zsh")
(jail.combinators.ro-bind "/home/${user}/.zshenv" "/home/${user}/.zshenv")
(jail.combinators.ro-bind "${pkgs.fzf}/share/fzf" "/usr/share/fzf")
(jail.combinators.ro-bind "/home/${user}/.nix-profile" "/home/${user}/.nix-profile")
];
in
{
apps.${system}.default = {
type = "app";
program = "${jailed-shell}/bin/zsh";
};
};
}
We use symlinkJoin to have all tools in single nix path and add read only finds to some home dir dotfiles. ofc pip won’t work, since it’s in nix store, but we can add a combinator that will spawn a virtualenv and activate it on some tmpfs mount. If I could only override nix shell with this it would be awesome 
1 Like
Are you just trying to enter a subshell with this? You can get packages in a subshell ad-hoc with nix shell, and it works with a package that was symlinkJoin’d.
Edit: Sorry, I misread, you already tried nix shell, thought maybe you were trying nix develop or nix-shell.
nix shell will run $SHELL after entering the subshell, and all devshell commands use bash. There isn’t a way to override that behavior unfortunately.
Thanks! Well I’ll have to use nix run then - seems like a reasonable tradeoff
1 Like
One thought, have you considered nix containers? It’s not as elegant, but you could do that and then use ssh to tunnel into the container? There may be other caveats, not sure.
Thanks! I did and I tested vms. I already have a thin qemu vm with blackarch that is spinned via a systemd unit, but I’m looking for something seamless. The idea is that during work I often test out various tools. When I get them to work I can then modify the flake to prepare a real nix package and make the tool persistent if it proved safe and valuable
My gripe with containers is that they require root. I was thinking I could extend this shell idea to have a non-bubblewrapped shell with separate history and goodies not to pollute main shell history with commands that contain credentials etc.
when you figure it out, drop a link 
Below is a working solution. It wraps zsh with bubblewrap and uses a combinator to inject creation of python virtual env along with activation. Virtualenv is created in tmpfs in /env dir so it does not persist after shell is closed. I used no-new-session combinator to allow fzf to do it’s TUI things to terminal.
Now the only option I’m missing it to pass all stuff from parent nix store to jail to avoid fixing myriad of issues like missing grep 
EDIT: added /run/current-system/sw/bin ro-bind and added this to $PATH and now jailed shell has access to all host’s software.
It will run with nix run so the workflow will look like this:
- clone some sus git repo like
github.com/sus-dude/free-dc-privesc-dude-i-swear
- copy the flake (or gen from template)
- nix run
- pip install -r totally_legit_deps.txt
- profit
{
description = "A very basic jail flake";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
jail-nix.url = "sourcehut:~alexdavid/jail.nix";
};
outputs =
{
self,
nixpkgs,
jail-nix,
...
}@inputs:
let
user = "myuser";
system = "x86_64-linux";
pkgs = import nixpkgs { inherit system; };
jail = jail-nix.lib.init pkgs;
python3-env = pkgs.python3.withPackages (py: [
# any packages from nixox is we want them outside of venv
py.virtualenv
]);
pentest-env = pkgs.symlinkJoin {
name = "pentest-env";
paths = [
python3-env
pkgs.zsh
pkgs.zsh-completions
pkgs.openssl # hate this crap, always makes me miserable :D
pkgs.coreutils
pkgs.gcc # often needed for pip compiling
pkgs.fzf
# pkgs.eza and so one
];
};
jailed-shell = jail "zsh" "${pentest-env}/bin/zsh" [
jail.combinators.network
jail.combinators.mount-cwd
# next two entries expose all software from outside of the container
(jail.combinators.ro-bind "/run/current-system/sw/bin" "/run/current-system/sw/bin")
(jail.combinators.add-path "/run/current-system/sw/bin")
# make zsh work with my config
(jail.combinators.add-path "${pentest-env}/bin")
(jail.combinators.readonly "/home/${user}/.zsh")
(jail.combinators.readonly "/home/${user}/.zshenv")
(jail.combinators.readonly "/etc/os-release")
(jail.combinators.readonly "/usr/bin")
(jail.combinators.ro-bind "${pkgs.fzf}/share/fzf" "/usr/share/fzf")
# tmpfs for temporary virtualenv
(jail.combinators.tmpfs "/env")
# needed for fzf to work
(jail.combinators.no-new-session)
# wrapper that initializes virtualenv prior to running shell
(jail.combinators.wrap-entry (entry: ''
echo "Launching jail..."
virtualenv /env
export VIRTUAL_ENV=/env
export PATH="/env/bin:$PATH"
exec ${entry}
''))
];
in
{
apps.${system}.default = {
type = "app";
program = "${jailed-shell}/bin/zsh";
};
};
}
1 Like
Neat work, I like and want to use more of jail.nix as I’ve only dabbled with it. I’m not convinced it’s the appropriate tool for the job but that’s up to your threat model. You get better protections with other mechanisms, especially VMs, which can be declaratively and simply made with microvm.nix. If you haven’t checked it out, you should. It could be a useful tool at some point in the future for your various stuff.
1 Like
Thanks for chiming in! Yeah namespaces are definitely nowhere near as safe as a vm. I’m exaggerating a bit with those ‘creepy’ github repos - also if this was not a targeted attack I don’t think the adversary would be able to hop out of the container in a limited timeframe when one is running the tool a couple of times. What I like about bubblewrap is that you can easily montior namespaces with sysdig to see weird traffic or disk activity outside cwd. I’m looking to make this a staging area for tools I want to package (ldapx, krbrelayx among others) so after engagement I can slowly exchange packages from virtual env with nixos equivalents). Python is the only part of nix that I find lacking. Many tools malfunction due to small differences in dependency versions in nix vs pip or issues with python packages that rely on some c-lib (like cursed openssl ).
I know about microvm, but I’m using my own-baked nix module with single tiny qemu vm managed via systemd unit - it’s for risky stuff or ugly stuff (anything that touches oracle and requires their proprietary bullshit libs in right places) and is designed more as ad-hoc spawn than declartive approach to minimize resistence of using them under pressure (I type ‘jk’ and it spawns a vm that is optimized for fast boot and immediately connects to it via ssh - subsequent ‘jk’ calls ssh directly and other scripts for easy snapshot management without ugly xml or clunky ui provided by typical qemu wrappers).
3 Likes
As long as you knew, thanks for sharing the original post, I’m gonna tuck that away for future reference since it is interesting
2 Likes
Very nice. I will be book marking this for future reference. If you do end up writing some kind of subshell command, I’d consider adding it to ns as a feature. You may also want to see if maintainers of nix shell/nix develop would consider allowing a shell override so this is possible without having to write your own subshell command.
3 Likes