Creating a docker container with devShell dependencies

I work on a team that uses Nix (on MacOS) to build the development environment for our python/django/react web application. This is done with the basic flake-utils and devShell approach. We use a flakes to ensure reproducibility across the organization. We do not use poetry2nix because the amount of overrides I need to do to install dependencies is a nightmare and I’ve never successfully got this to work. But I’ll save that for a different post.

flake.nix

{
  description = "A Very Nice Flake";
  inputs = {
    nixpkgs = {
      url = "github:NixOS/nixpkgs/nixos-unstable";
    };
    flake-utils = {
      url = "github:numtide/flake-utils";
    };
  };
  outputs = { self, nixpkgs, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          config = { };
        };
      in
      {
        devShells.default = pkgs.mkShell {
          packages = with pkgs; [
            nodejs_20
            yarn
            python311
            postgresql_13 # needed for some DB service commands
            poetry
            chromium
            chromedriver
            sops
            # there's more but I'm keeping it brief here
          ];
        };
      }
    );
}

No issues here :slight_smile:

Other than poetry2nix, flakes for development environments are going great. However, we’re still using classic docker compose and a python:3-11-slim-buster base for our CI pipeline. Recently, I have been bitten by small differences in versions between our development and production environment. My current goal is to unify the build so that our production docker container for the web app has the exact same dependencies as the development shell.

I am trying to start with a basic implementation but I’m having a heck of a time getting this to work. My thought was that I could use the nixos/nix docker container as a base, drop in the flake for our development shell and be off to the races. So I have something like this…

Dockerfile

FROM nixos/nix:latest 

COPY flake.nix flake.lock poetry.lock pyproject.toml /tmp/build/
WORKDIR /tmp/build

RUN nix develop \
    --extra-experimental-features "nix-command flakes" 

ENV POETRY_VIRTUALENVS_PATH=/tmp/python
RUN poetry install
ENV PYTHONPATH=/tmp/python

entrypoint ["/bin/sh"]

NOTE: This isn’t a polished implementation, I am aware. For example, I need to specify the python path better and it shouldn’t be in the tmp directory.

The problem I’m running into here is that since the development shell opens a new process, we cannot find poetry in the next command. The seemingly basic thing I’d like to do here is have the devShell dependencies installed at the container level. I’m hopeful someone knows of a basic approach to load the devShell output into the current system profile so that I have access to the dependencies without having to call nix develop

❯ docker build --tag test-nix . 
[+] Building 47.5s (9/9) FINISHED                                                                                                                                 docker:desktop-linux
 => [internal] load .dockerignore                                                                                                                                                 0.0s
 => => transferring context: 2B                                                                                                                                                   0.0s
 => [internal] load build definition from Dockerfile                                                                                                                              0.0s
 => => transferring dockerfile: 1.37kB                                                                                                                                            0.0s
 => [internal] load metadata for docker.io/nixos/nix:latest                                                                                                                       0.9s
 => [internal] load build context                                                                                                                                                 0.0s
 => => transferring context: 127B                                                                                                                                                 0.0s
 => CACHED [1/5] FROM docker.io/nixos/nix:latest@sha256:935cdafedba76f86c45d01f30f243474c958e1099029b292e5f2fbeac5f8805e                                                          0.0s
 => [2/5] COPY flake.nix flake.lock poetry.lock pyproject.toml /tmp/build/                                                                                                        0.0s
 => [3/5] WORKDIR /tmp/build                                                                                                                                                      0.0s
 => [4/5] RUN nix develop     --extra-experimental-features "nix-command flakes"                                                                                                 46.0s
 => ERROR [5/5] RUN poetry install                                                                                                                                                0.4s
------                                                                                                                                                                                 
 > [5/5] RUN poetry install:                                                                                                                                                           
0.390 /bin/sh: line 1: poetry: command not found                                                                                                                                       
------                                                                                                                                                                                 
Dockerfile:10                                                                                                                                                                          
--------------------
   8 |     
   9 |     ENV POETRY_VIRTUALENVS_PATH=/tmp/python
  10 | >>> RUN poetry install
  11 |     ENV PYTHONPATH=/tmp/python
  12 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c poetry install" did not complete successfully: exit code: 127

Maybe this is a foolish approach? I’m sure there are people who are going to read this that tell me I should just build the container with Nix. I’m willing to try that later but if I could just get our web container to be based on Nix with all dependencies pinned to the same version used in development, that would be a lot easier to plug into the existing infrastructure.