I work on a team that uses Nix (on MacOS) to build the development environment for our python/django/react web application. This is done with the basic flake-utils and devShell approach. We use a flakes to ensure reproducibility across the organization. We do not use poetry2nix because the amount of overrides I need to do to install dependencies is a nightmare and I’ve never successfully got this to work. But I’ll save that for a different post.
flake.nix
{
description = "A Very Nice Flake";
inputs = {
nixpkgs = {
url = "github:NixOS/nixpkgs/nixos-unstable";
};
flake-utils = {
url = "github:numtide/flake-utils";
};
};
outputs = { self, nixpkgs, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
config = { };
};
in
{
devShells.default = pkgs.mkShell {
packages = with pkgs; [
nodejs_20
yarn
python311
postgresql_13 # needed for some DB service commands
poetry
chromium
chromedriver
sops
# there's more but I'm keeping it brief here
];
};
}
);
}
No issues here
Other than poetry2nix, flakes for development environments are going great. However, we’re still using classic docker compose and a python:3-11-slim-buster base for our CI pipeline. Recently, I have been bitten by small differences in versions between our development and production environment. My current goal is to unify the build so that our production docker container for the web app has the exact same dependencies as the development shell.
I am trying to start with a basic implementation but I’m having a heck of a time getting this to work. My thought was that I could use the nixos/nix docker container as a base, drop in the flake for our development shell and be off to the races. So I have something like this…
Dockerfile
FROM nixos/nix:latest
COPY flake.nix flake.lock poetry.lock pyproject.toml /tmp/build/
WORKDIR /tmp/build
RUN nix develop \
--extra-experimental-features "nix-command flakes"
ENV POETRY_VIRTUALENVS_PATH=/tmp/python
RUN poetry install
ENV PYTHONPATH=/tmp/python
entrypoint ["/bin/sh"]
NOTE: This isn’t a polished implementation, I am aware. For example, I need to specify the python path better and it shouldn’t be in the tmp directory.
The problem I’m running into here is that since the development shell opens a new process, we cannot find poetry in the next command. The seemingly basic thing I’d like to do here is have the devShell dependencies installed at the container level. I’m hopeful someone knows of a basic approach to load the devShell output into the current system profile so that I have access to the dependencies without having to call nix develop
❯ docker build --tag test-nix .
[+] Building 47.5s (9/9) FINISHED docker:desktop-linux
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.37kB 0.0s
=> [internal] load metadata for docker.io/nixos/nix:latest 0.9s
=> [internal] load build context 0.0s
=> => transferring context: 127B 0.0s
=> CACHED [1/5] FROM docker.io/nixos/nix:latest@sha256:935cdafedba76f86c45d01f30f243474c958e1099029b292e5f2fbeac5f8805e 0.0s
=> [2/5] COPY flake.nix flake.lock poetry.lock pyproject.toml /tmp/build/ 0.0s
=> [3/5] WORKDIR /tmp/build 0.0s
=> [4/5] RUN nix develop --extra-experimental-features "nix-command flakes" 46.0s
=> ERROR [5/5] RUN poetry install 0.4s
------
> [5/5] RUN poetry install:
0.390 /bin/sh: line 1: poetry: command not found
------
Dockerfile:10
--------------------
8 |
9 | ENV POETRY_VIRTUALENVS_PATH=/tmp/python
10 | >>> RUN poetry install
11 | ENV PYTHONPATH=/tmp/python
12 |
--------------------
ERROR: failed to solve: process "/bin/sh -c poetry install" did not complete successfully: exit code: 127
Maybe this is a foolish approach? I’m sure there are people who are going to read this that tell me I should just build the container with Nix. I’m willing to try that later but if I could just get our web container to be based on Nix with all dependencies pinned to the same version used in development, that would be a lot easier to plug into the existing infrastructure.