Canonical Flake for Full Stack Web App (Purescript/Haskell/PostgreSQL)

I am working on a FRP web app that runs Purescript on the front end and Haskell (Servant and Opaleye) on the back end talking to a PostgreSQL database.

Right now, PostgreSQL runs natively (I found a flake containing a set of shell scripts from the Elixir discourse that help me start, stop, setup, etc the database) but I can’t help but have the inkling that I’m going about this all wrong.

My instincts say that is far more advisable to do this as a Docker or OCI container (despite my Nixy aversion to those solutions). I don’t think I want to go the route of using postgresql as a NixOS service for the same portability/canonical instincts.

Here’s a link to the relevant branch’s flake:
my current flawed, naive full stack implementation


My question:

My instincts say that I should instead be creating an OCI image that spins up the database rather than running it natively. Is that true?

Can you point me to a canonical example or some documentation that would set me straight on this type of thing? Obviously I tend to do things using 100% Nix but this one has me a little confused since it feels wrong to use Nix for this type of thing.

Basically, I want my entire dev environment to be provisioned and spun up using that one Nix flake. I’m doing this to not only provision my dev environment anywhere but to also deploy this app easily when that time comes. If someone doesn’t mind steering me straight, I’d be incredibly thankful.

Bonus Issue I’ve been struggling with: My preferred Haskell build tool (IOG’s Haskell.nix) is very broken right now with the recent major changes to postgresql-libpq. I get lots of errors when I run ‘nix-develop’ in the build which refer to pkg-config not knowing where to find anything with the recent changes. I tried setting env variables in the shell hook but that really never worked. So, I just commented it out and use ‘cabal build’ at the moment until I can possibly fix that.

ps. I also have had similar issues with building Purescript with purs-nix and have abandoned it since the new version of spago was launched.

This reply may not answer your question. But hopefully it gives you some useful trains of though to follow when coming up with a solution for yourself.

You posted this just before I was going to try deploying a webapp with NixOS and Docker, so in a week or so after I try do that I can give you more info from my experience.

But anyway a couple of my opinions:

A database is stateful, ugly, and “impure” no matter what. Nix is built around the concept of being able to throw away state and recreate it easily. So when you combine the two it is like putting a square peg in a round hole.

In a situation like this your instincts are right that you should be using another tool like OCI containers. In my NixOS config I have this systemd service which runs an OCI container via Podman (yes docker-compose can use Podman instead of Docker). In this case I start it with a systemd service but of course as a developer you could just run the composeScript it in a shellHook in a devShell.

In your compose.yaml, as long as you specify your container to be in a remote repo, it will just automatically pull it from there and reproducibly run the same startup commands. No need to have any preexisting state on your system. So it is about as declarative as Ansible which, although it is not Nix, is still pretty good.

Here is a sanitized systemd service from my config:

  systemd.services.docker-compose-my-service = (
    let
      composeScript = pkgs.writeScript "compose_script" ''
        #!/bin/sh

        export PODMAN_COMPOSE_PROVIDER=${pkgs.docker-compose}/bin/docker-compose

        ${pkgs.podman}/bin/podman compose --file /path/to/compose.yaml up --remove-orphans --force-recreate --detach

      '';

    in
    {
	enable = true;
      description = "Start my-service via docker-compose";
      wants = [ "network-online.target" ];
      after = [ "network-online.target" ];
      wantedBy = [ "multi-user.target" ];
      serviceConfig = {
        Type = "oneshot";
        ExecStart = "-${composeScript}";
      };
    }
  );

And don’t forget that you can write your Dockerfiles in Nix with dockerTools and then pull them in to your docker-compse. Building and running Docker images — nix.dev documentation

IDK if there is a Nix equivalent for docker-compose.

1 Like

Thanks Zach! I’m loving the variety of opinions I’ve gotten so far on this. I cross-posted this question to the Purescript and Haskell discourses and got a good reply on the Haskell one about copying IHP which seems to have chosen to run it as a NixOS service.

I’ll report back with my progress.

I managed to get the service option working really well and reliably.

Here’s what the service module I made looks like:

{ config, lib, pkgs, name, ... }:

with lib;
let
  cfg = config.services.${pgConfig.database.name}.postgresql;
  pgConfig = import ./postgresql-config.nix;

in {

  options.services.${pgConfig.database.name}.postgresql = {
    enable = mkEnableOption "Cheeblr PostgreSQL Service";
    package = mkOption {
      type = types.package;
      default = pkgs.postgresql;
      description = "PostgreSQL package to use";
    };
    port = mkOption {
      type = types.port;
      default = pgConfig.database.port;
      description = "PostgreSQL port number";
    };
    dataDir = mkOption {
      type = types.str;
      default = "/var/lib/postgresql/${config.services.postgresql.package.psqlSchema}";
      description = "PostgreSQL data directory";
    };
  };

  config = mkIf cfg.enable {
    services.postgresql = {
      enable = true;
      package = cfg.package;
      enableTCPIP = true;
      port = cfg.port;
      dataDir = cfg.dataDir;
      ensureDatabases = [ pgConfig.database.name ];
      
      authentication = pkgs.lib.mkOverride 10 ''
        # Local connections use password
        local   all             all                                     trust
        # Allow localhost TCP connections with password
        host    all             all             127.0.0.1/32           trust
        host    all             all             ::1/128                trust
      '';

      initialScript = pkgs.writeText "${pgConfig.database.name}-init" ''
        DO $$
        BEGIN
          IF NOT EXISTS (SELECT FROM pg_user WHERE usename = '${pgConfig.database.user}') THEN
            CREATE USER ${pgConfig.database.user} WITH PASSWORD '${pgConfig.database.password}' SUPERUSER;
          END IF;
        END
        $$;

        CREATE DATABASE ${pgConfig.database.name};
        GRANT ALL PRIVILEGES ON DATABASE ${pgConfig.database.name} TO ${pgConfig.database.user};
      '';

      settings = {
        # Default config
        max_connections = 100;
        shared_buffers = "128MB";
        dynamic_shared_memory_type = "posix";
        log_destination = "stderr";
        logging_collector = true;
        log_directory = "log";
        log_filename = "postgresql-%Y-%m-%d_%H%M%S.log";
        log_min_messages = "info";
        log_min_error_statement = "info";
        log_connections = true;
      };
    };

    environment.systemPackages = [ cfg.package ];
    
    environment.variables = {
      PGHOST = "localhost";
      PGPORT = toString cfg.port;
      PGUSER = pgConfig.database.user;
      PGDATABASE = pgConfig.database.name;
      DATABASE_URL = "postgresql://${pgConfig.database.user}:${pgConfig.database.password}@localhost:${toString cfg.port}/${pgConfig.database.name}";
    };
  };
}

and I broke the config out to one file so I can use the settings in many different iterations of postgresql:

{ ... }: {
  database = {
    name = "cheeblr";
    user = "postgres";
    password = "postgres";
    port = 5432;
    dataDir = "./postgresql";
    settings = {
      max_connections = 100;
      shared_buffers = "128MB";
      dynamic_shared_memory_type = "posix";
      log_destination = "stderr";
      logging_collector = true;
      log_directory = "log";
      log_filename = "postgresql-%Y-%m-%d_%H%M%S.log";
      log_min_messages = "info";
      log_min_error_statement = "info";
      log_connections = true;
      listen_addresses = "localhost";
    };
  };
}

and here are the utilities I built to work with the database:

{ pkgs
, lib ? pkgs.lib
, name ? "cheeblr"
}: 

let
  pgConfig = import ./postgresql-config.nix { };
  
  postgresql = pkgs.postgresql;
  bin = {
    pgctl = "${postgresql}/bin/pg_ctl";
    psql = "${postgresql}/bin/psql";
    initdb = "${postgresql}/bin/initdb";
    createdb = "${postgresql}/bin/createdb";
    pgIsReady = "${postgresql}/bin/pg_isready";
  };

  config = {
    dataDir = pgConfig.database.dataDir;
    port = pgConfig.database.port;
    user = pgConfig.database.user;
    password = pgConfig.database.password;
  };

  mkPgConfig = ''
listen_addresses = '${pgConfig.database.settings.listen_addresses}'
port = ${toString config.port}
unix_socket_directories = '$PGDATA'
max_connections = ${toString pgConfig.database.settings.max_connections}
shared_buffers = '${pgConfig.database.settings.shared_buffers}'
dynamic_shared_memory_type = '${pgConfig.database.settings.dynamic_shared_memory_type}'
log_destination = 'stderr'
logging_collector = off
'';

  mkHbaConfig = ''
local   all             all                                     trust
host    all             all             127.0.0.1/32           trust
host    all             all             ::1/128                trust
'';

  envSetup = ''
    export PGPORT="''${PGPORT:-${toString config.port}}"
    export PGUSER="''${PGUSER:-${config.user}}"
    export PGDATABASE="''${PGDATABASE:-${pgConfig.database.name}}"
    export PGHOST="$PGDATA"
  '';

  validateEnv = ''
    if [ -z "$PGDATA" ]; then
      echo "Error: PGDATA environment variable must be set"
      exit 1
    fi
  '';

in {
  inherit config;

  setupScript = pkgs.writeShellScriptBin "pg-setup" ''
    ${envSetup}
    ${validateEnv}

    init_database() {
      echo "Creating PGDATA directory at: $PGDATA"
      rm -rf "$PGDATA"
      mkdir -p "$PGDATA"

      echo "Initializing database..."
      ${bin.initdb} -D "$PGDATA" \
            --auth=trust \
            --no-locale \
            --encoding=UTF8 \
            --username="${config.user}"

      # Write config files exactly as in working version
      cat > "$PGDATA/postgresql.conf" << EOF
${mkPgConfig}
EOF

      cat > "$PGDATA/pg_hba.conf" << EOF
${mkHbaConfig}
EOF
    }

    start_database() {
      echo "Starting PostgreSQL..."
      ${bin.pgctl} -D "$PGDATA" -l "$PGDATA/postgresql.log" start

      if [ $? -ne 0 ]; then
        echo "PostgreSQL failed to start. Here's the log:"
        cat "$PGDATA/postgresql.log"
        return 1
      fi

      echo "Waiting for PostgreSQL to be ready..."
      RETRIES=0
      while ! ${bin.pgIsReady} -h "$PGHOST" -p "$PGPORT" -q; do
        RETRIES=$((RETRIES+1))
        if [ $RETRIES -eq 10 ]; then
          echo "PostgreSQL failed to become ready. Here's the log:"
          cat "$PGDATA/postgresql.log"
          return 1
        fi
        sleep 1
        echo "Still waiting... (attempt $RETRIES/10)"
      done
    }

    setup_database() {
      echo "Creating database..."
      ${bin.createdb} -h "$PGHOST" -p "$PGPORT" "$PGDATABASE"
      
      if [ $? -ne 0 ]; then
        echo "Failed to create database"
        return 1
      fi

      # Use DO block for conditional user creation
      ${bin.psql} -h "$PGHOST" -p "$PGPORT" "$PGDATABASE" << EOF
DO \$\$
BEGIN
  IF NOT EXISTS (SELECT FROM pg_user WHERE usename = '${config.user}') THEN
    CREATE USER ${config.user} WITH PASSWORD '${config.password}' SUPERUSER;
  END IF;
END
\$\$;
GRANT ALL PRIVILEGES ON DATABASE ${pgConfig.database.name} TO ${config.user};
EOF
    }

    cleanup() {
      if [ -f "$PGDATA/postmaster.pid" ]; then
        echo "Stopping PostgreSQL..."
        ${bin.pgctl} -D "$PGDATA" stop -m fast
      fi
    }

    trap cleanup EXIT

    init_database && start_database && setup_database

    echo "Development environment ready:"
    echo "  Socket directory: $PGHOST"
    echo "  Port: $PGPORT"
    echo "  Database URL: postgresql://${config.user}:${config.password}@localhost:$PGPORT/$PGDATABASE"
    echo ""
    echo "You can connect to the database using:"
    echo "  ${bin.psql} -h $PGHOST -p $PGPORT $PGDATABASE"
  '';

  pg-start = pkgs.writeShellScriptBin "pg-start" ''
    ${envSetup}
    ${validateEnv}

    echo "Starting PostgreSQL..."
    ${bin.pgctl} -D "$PGDATA" -l "$PGDATA/postgresql.log" start

    if [ $? -ne 0 ]; then
      echo "PostgreSQL failed to start. Here's the log:"
      cat "$PGDATA/postgresql.log"
      exit 1
    fi

    echo "Waiting for PostgreSQL to be ready..."
    RETRIES=0
    while ! ${bin.pgIsReady} -h "$PGHOST" -p "$PGPORT" -q; do
      RETRIES=$((RETRIES+1))
      if [ $RETRIES -eq 10 ]; then
        echo "PostgreSQL failed to become ready. Here's the log:"
        cat "$PGDATA/postgresql.log"
        exit 1
      fi
      sleep 1
      echo "Still waiting... (attempt $RETRIES/10)"
    done
  '';

  pg-connect = pkgs.writeShellScriptBin "pg-connect" ''
    ${envSetup}
    ${validateEnv}

    if [ -z "$PGPORT" ]; then
      echo "Port must be set"
      exit 1
    fi
    if [ -z "$PGDATABASE" ]; then
      echo "Database name must be set"
      exit 1
    fi
    ${bin.psql} -h $PGHOST -p $PGPORT $PGDATABASE
  '';

  pg-stop = pkgs.writeShellScriptBin "pg-stop" ''
    ${envSetup}
    ${validateEnv}
    ${bin.pgctl} -D "$PGDATA" stop -m fast
  '';
}

and the current state of my flake:

{
  description = "cheeblr";

  inputs = {
    # IOG inputs
    iogx = {
      url = "github:input-output-hk/iogx";
      inputs.hackage.follows = "hackage";
      inputs.CHaP.follows = "CHaP";
      inputs.nixpkgs.follows = "nixpkgs";
    };

    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";

    iohkNix = {
      url = "github:input-output-hk/iohk-nix";
      inputs.nixpkgs.follows = "nixpkgs";
    };

    hackage = {
      url = "github:input-output-hk/hackage.nix";
      flake = false;
    };

    CHaP = {
      url = "github:IntersectMBO/cardano-haskell-packages?rev=35d5d7f7e7cfed87901623262ceea848239fa7f8";
      flake = false;
    };

    purescript-overlay = {
      url = "github:harryprayiv/purescript-overlay";
      inputs.nixpkgs.follows = "nixpkgs";
    };
        
    flake-utils.url = "github:numtide/flake-utils";
    flake-compat = {
      url = "github:edolstra/flake-compat";
      flake = false;
    };
  };

  outputs = { self, nixpkgs, flake-utils, iohkNix, CHaP, iogx, purescript-overlay, ... }:
    {
      nixosModules = {
        postgresql = import ./nix/postgresql-service.nix;
        default = { ... }: {
          imports = [ self.nixosModules.postgresql ];
        };
      };
    } // flake-utils.lib.eachSystem ["x86_64-linux" "x86_64-darwin" "aarch64-darwin"] (system: let
      
      name = "cheeblr";
      lib = nixpkgs.lib;

      overlays = [
        iohkNix.overlays.crypto
        purescript-overlay.overlays.default
      ];
      
      pkgs = import nixpkgs {
        inherit system overlays;
      };

      # Shell apps
      postgresModule = import ./nix/postgres-utils.nix {
        inherit pkgs name;
      };

      vite = pkgs.writeShellApplication {
        name = "vite";
        runtimeInputs = with pkgs; [ nodejs-slim ];
        text = ''
          export CHEEBLR_BASE_PATH="${self}"
          npx vite --open
        '';
      };

      concurrent = pkgs.writeShellApplication {
        name = "concurrent";
        runtimeInputs = with pkgs; [ concurrently ];
        text = ''
          concurrently\
            --color "auto"\
            --prefix "[{command}]"\
            --handle-input\
            --restart-tries 10\
            "$@"
        '';
      };

      spago-watch = pkgs.writeShellApplication {
        name = "spago-watch";
        runtimeInputs = with pkgs; [ entr spago-unstable ];
        text = ''find {src,test} | entr -s "spago $*" '';
      };

      code-workspace = pkgs.writeShellApplication {
        name = "code-workspace";
        runtimeInputs = with pkgs; [ vscodium ];
        text = ''
          codium cheeblr.code-workspace
        '';
      };

      dev = pkgs.writeShellApplication {
        name = "dev";
        runtimeInputs = with pkgs; [
          nodejs-slim
          spago-watch
          vite
          concurrent
        ];
        text = ''
          concurrent "spago-watch build" vite
        '';
      };
      
    in {
      legacyPackages = pkgs;

      devShell = pkgs.mkShell {
        inherit name;
        
        nativeBuildInputs = with pkgs; [
          pkg-config
          postgresql
          zlib
          openssl.dev
          libiconv
          openssl
        ];

        buildInputs = with pkgs; [
          # Front End tools
          esbuild
          nodejs_20
          nixpkgs-fmt
          purs
          purs-tidy
          purs-backend-es
          purescript-language-server
          spago-unstable
      
          # Back End tools
          cabal-install
          ghc
          haskellPackages.fourmolu
          haskell-language-server
          hlint
          zlib
          pgcli
          pkg-config
          openssl.dev
          libiconv
          openssl
          
          # PostgreSQL tools
          postgresModule.setupScript 
          postgresModule.pg-start
          postgresModule.pg-connect
          postgresModule.pg-stop
          
          pgadmin4-desktopmode
          # pgmanage
          # pgadmin4

          # DevShell tools
          spago-watch
          vite
          dev
          code-workspace
          
        ] ++ (pkgs.lib.optionals (system == "aarch64-darwin")
          (with pkgs.darwin.apple_sdk.frameworks; [
            Cocoa
            CoreServices
          ]));
        shellHook = ''
                  # Set up PostgreSQL environment
                  export PGDATA="$PWD/.postgres"
                  export PGPORT="5432"
                  export PGUSER="postgres"
                  export PGPASSWORD="postgres"
                  export PGDATABASE="${name}"

                  # Run the setup script
                  pg-setup
                '';
      };
    });

  nixConfig = {
    extra-experimental-features = ["nix-command flakes" "ca-derivations"];
    allow-import-from-derivation = "true";
    extra-substituters = [
...
    ];
  };
}

In my NixOS config I have this systemd service which runs an OCI container via Podman (yes docker-compose can use Podman instead of Docker). In this case I start it with a systemd service but of course as a developer you could just run the composeScript it in a shellHook in a devShell.

In your compose.yaml, as long as you specify your container to be in a remote repo, it will just automatically pull it from there and reproducibly run the same startup commands. No need to have any preexisting state on your system. So it is about as declarative as Ansible which, although it is not Nix, is still pretty good.

I’m now looking directly at this comment when I think through going the OCI route. I may have a few things up my sleeve about drawing those environment variables from the config file I made.

I’ve gotten a lot better at passing values around the Nix structure recently, so I figure why not? I could build some Nixy declarations into it and perhaps dynamically derive parts of that compose.yaml during the shell hook perhaps. The more I think about it, the worse it sounds. :rofl:a chicken or egg problem. “If you create the compose file who creates the compose file if you can’t be there, Nix?”

I’m soon going to embarking on a similar route (I think I’ll probably go with Hyperbole or Miso instead of purescript though) and I plan on running my application natively with nixos services but you might also be interested in Services Flake:

I haven’t personally used it but it seems to fit the feature set you’re looking for.

1 Like