Hi,
I create a simple flake that compile my program application using mkPoetryApplication given the poetry file.
But at runtime i need some other packages to run the program :
What’s the canonical way to add system dependencies into my current flakes ?
Ps : I don’t really understand the difference between mkPoetryApplication and mkPoetryEnv and how the two are related ?
I don’t know if it’s the canonical way but …
I first found nativeBuildInputs = option but at runtime pkgs are not found.
{
description = "Application packaged using poetry2nix";
inputs.flake-utils.url = "github:numtide/flake-utils";
inputs.nixpkgs.url = "github:NixOS/nixpkgs";
inputs.poetry.url = "github:nix-community/poetry2nix";
outputs = inputs@{ self, nixpkgs, flake-utils, poetry }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs { inherit system; overlays = [ poetry.overlay ]; };
inherit (pkgs) poetry2nix;
in {
defaultPackage = poetry2nix.mkPoetryApplication {
projectDir = ./.;
python = pkgs.python39;
nativeBuildInputs = [
pkgs.pandoc
pkgs.bash
pkgs.wget
pkgs.findutils
];
};
}
);
}
Most of these build scripts uses stdenv under the hood, so take a look at https://github.com/NixOS/nixpkgs/blob/dc7c656444f0691c5ecd604cc1b4af4b02580645/doc/stdenv/stdenv.chapter.md#specifying-dependencies-ssec-stdenv-dependencies
There’s a lot of different types of dependencies, especially if you want to bother with cross-compilation. Since you’re doing a poetry project though, you’re probably better off just sticking to these general rules for same-architecture dependencies:
nativeBuildInputs for build dependencies only, stuff you need to build the package but not to run it.buildInputs for build and runtime dependencies, stuff you need to build the package and to run it.propagatedBuildInputs for build dependencies that people will need when they’re using your package as a library later-on.At least, this is my basic understanding of it, and the three most common types I’ve seen.
Basically, try putting those packages in buildInputs instead.
As for the difference between mkPoetryApplication and mkPoetryEnv, the first will give you just a binary that under-the-hood uses poetry and python, but just provides the user something they run, while the second will give you a python REPL with the dependencies built.
I’ve seen the first used primarily for standalone apps, and the second for specific environment tools, like running Django on an Nginx Unit server.
Thanks for this reply, perhaps i misunderstand something, so i add some new informations to this post :
image: nixpkgs/nix-flakes:latest
stages:
- build
- publish
process:
stage: build
before_script:
- nix build
script:
- sh build.sh
artifacts:
paths:
- notes_converted/
expire_in: 1 week
pages:
stage: publish
script:
- cp -r notes_converted/public/ public/
artifacts:
paths:
- public
only:
- main
The gitlabrunner say :
$ nix buildwarning: Git tree '/builds/gt-notebook/wiki' is dirty$ sh build.shbuild.sh: line 4: grep: command not foundbuild.sh: line 4: xargs: command not foundAnd later pandoc not found.
So my mistake here was perhaps to add this dependency into the flake output and not into the docker nixos/nix-flakes used by gitlab runner ?
Ah, I see, you’re trying to make a polyglot ci system based off Docker, GitLab, and Nix. So you guessed right, the mistake you made is not building your own Docker image on top of the nixpkgs/nix-flakes:latest image where you add those dependencies to the Docker image with nix-env, since you don’t need them when you’re building or running your app, but to move your app around and run scripts on your built app.
That way, when your scripts run, they should find those dependencies on the path.
Try looking instead at GitHub - LnL7/nix-docker: Docker images for the Nix package manager for making a Docker image with the dependencies you need. I think that will fit into your GitLab build system better.
I also try “nix develop” to see … but that not work. So i try the github link you sent me to see if it’s works better for my use case 
Finally i found (following your tips @gh0st), the good command for runtime (not python) external program/dependency, this is propagatedBuildInputs
propagatedBuildInputs = [
pkgs.pandoc
pkgs.graphviz
pkgs.wget
];
I continue to post my result here, perhaps that interest other nix haskell/flakes newbies like me …
To be short, after @gh0st answer me, i finally found that pkgs.pandoc is not updated with the latest version … or … i need an updated version for my python package.
I try with haskellpackages.pandoc_xxx… on more recent version, but they need some packages/dependencies not available in hackage.
SO i take the road to create a flake package that compile latest pandoc
I update my StackOverflow question with some result, using Haskell.nix
The flake :
{
description = "PandocProject";
inputs.haskellNix.url = "github:input-output-hk/haskell.nix";
inputs.nixpkgs.follows = "haskellNix/nixpkgs-unstable";
inputs.flake-utils.url = "github:numtide/flake-utils";
outputs = { self, nixpkgs, flake-utils, haskellNix }:
flake-utils.lib.eachSystem [ "x86_64-linux" ] (system:
let
overlays = [ haskellNix.overlay
(final: prev: {
# This overlay adds our project to pkgs
pandocProject = final.haskell-nix.cabalProject {
cabalProjectFreeze = null;
cabalProject = null;
cabalProjectLocal = null;
compiler-nix-name = "ghc8104";
src = final.haskell-nix.cleanSourceHaskell {
src = ./.;
name = "pandocProject";
};
pkg-def-extras = with final.haskell.lib; [];
};
})
];
pkgs = import nixpkgs { inherit system overlays; };
flake = pkgs.pandocProject.flake {};
in flake // {
# Built by `nix build .`
defaultPackage = flake.packages."pandoc:exe:pandoc";
});
}
Now i need to link the two flakes (pandoc → input → poetry2nix project), using cachix/cache because pandoc is heavy to compile.
I tried using nix flakes with poetry2nix, and i see that all the dependencies are being build. But when I run nix develop .#mkreports, I only get a python without any packages installed.
my flake
{
description = "Application packaged using poetry2nix";
inputs = {
flake-utils.url = "github:numtide/flake-utils";
nixpkgs.url = "nixpkgs";
poetry2nix = {
url = "github:nix-community/poetry2nix";
inputs.nixpkgs.follows = "nixpkgs";
};
};
outputs = { self, nixpkgs, flake-utils, poetry2nix }:
{
# Nixpkgs overlay providing the application
overlay = nixpkgs.lib.composeManyExtensions [
poetry2nix.overlay
(final: prev: {
# The application
mkreports = prev.poetry2nix.mkPoetryEnv {
projectDir = ./.;
editablePackageSources = {
mkreports = ./mkreports;
};
};
})
];
} // (flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [ self.overlay ];
};
in
rec {
#devShells = {
# mkreports = pkgs.mkreports;
#};
packages = {
mkreports = pkgs.mkreports;
};
# devShell = devShells.mkreports;
}));
}
I also tried with devShell not commented out. Am pretty new at this, any help appreciated. Thanks
Addendum: I found out when i build it, i get a python with everything installed. But I don’t understand why not when i do nix develop.
mkPoetryEnv is to build an environment used for development.
You probably want to use mkPoetryApplication or how it was called.
I tried that as well, with no success. mkPoetryEnv in the end at least gives me the editable env when I build it. and it is useable this way. Maybe I need to wrap this in another mkShell or something?
What exactly do you want to achieve?
run nix develop .#mypackage and then be dropped into a shell, where the PATH has been set correctly so that when I call python, the python version with all installed dependencies is available in the shell.
Should work with either mkPoetryEnv and mkPoetryApplication, where the former also provides the application in editable mode and the former only the dependencies.
Ok - it does. Was a mistake of mine. I had source ~/.nix-profile/bin in my home-manager bashrc instead of profile. On initiating nix-shell, the ~/.nix-profile/bin was added to the front of my path, with python installed there then shadowing the version that was available in my shell. Thanks for your help anyway and sorry about the mistake.
Hi, apologies for reviving this. I’ve stumbled across this trying to find a defintion for mkPoetryEnv and mkPoetryApplication and I’m a little confused - should that first former be a latter?
No, the latter “former” should be a “latter”.
To be sure that I don’t get it wrong (again), *Env provides the app in “editable mode” as well as all the required dependencies, including “dev only”, while *Application only provides the application
That’s a much clearer definition than that given by the README. Thank you very much!