Feedback on nix example project

Hi there,

(TLDR: this is a request for feedback)

Project link: GitHub - tfc/nix_cmake_example: An example with nix: How to automatically and reproducibly build and integration-test a database C++ app in 60 different build-configurations (library exchange/compiler exchange/static-nonstatic)

i created a nix example project that contains a simple C++ app and a python app. The C++ app uses postgres to insert (meaningless) example messages into the DB - the python app provides a simple web service to display what’s in the DB.

The project shall demonstrate how to:

  • build this reproducibly (with pinned nixpkgs)
  • build a dynamic and static version of the c++ app
  • build the c++ app with gcc and clang
  • perform an integration test with a VM that sets up a postgres server and then runs the apps.
  • later: how to insert dependencies with different versions.
  • how to run this on a hydra server

I would like to demonstrate this at my local C++ user group.
As i consider myself being a nix novice, it would be totally great if someone more experienced would have a look at it in order to point out stupid/wrong things, before i demonstrate it to other people, so i can fix them and learn from this.

i.e.

  • the form of the default.nix file - would more experienced people do that, too?
  • the static-magic in the release.nix file - is this good practise or gone bad somehow?

Thank you!

3 Likes

Did you forget to link to it?

Wow, you’re right. Thanks, sorry. :smiley: i added the link!

First of all: great that you’re spreading the love :wink:

Overall the structure looks good, a really nice and complete example — good work! I can suggest some little improvements in the details:

  • Rather than applying the package functions (import ./server and import ./client) manually as you are doing in release.nix, you could use callPackage like nixpkgs does itself to fill in the arguments for your package functions. This allows the function pattern to be a fair bit shorter while also making the dependencies more explicit — for the server component, this might look like { stdenv, libpqxx, boost, cmake, static ? false }: instead of providing default values for all the arguments.
    Using callPackage also results in the override function being added, which allows modifying the composition after the fact, making a static clang-built version look like mdb-server.override { static = true; stdenv = pkgs.clangStdenv; } instead.
  • You may want to use filterSource when importing the sources from a local directory, to leave out some unnecessary files — the current way will result in potentially unnecessary rebuilds, e.g. when a result symlink in one of the directories is updated or default.nix is modified in a way that wouldn’t otherwise affect the build. nixpkgs includes lib.cleanSource as a convenient starting point for building your own filter, or you can just use cleanSource directly if it matches your needs well enough.
1 Like

I think it’s a nice idea and a good “minimal working example” that covers also some not really clear areas like automated testing of applications.

Maybe a README that covers the very basics would be helpful to jumpstart and that shows how things can be composed together and used on their own as well like: to compile the server launch this command, to run the tests launch this other command and so on

2 Likes

@lheckemann thank you!
I created a branch where callPackage is used as you suggested: GitHub - tfc/nix_cmake_example at callpackage

Unfortunately, now it seems like i cannot go into the server directory and run nix-shell any longer in order to have the toolchain loaded.
Now it seems like nix-shell ../release.nix -A mdb-server does the job. Is this the suggested workflow in this case?

I will for sure have a look at filterSource, thanks for the suggestion, i did not know about this, yet!

@zarel yes, you are right. A readme is a must! I was going to write one as soon as i am sure that i don’t have to rewrite everything after the feedback-round here.

Thanks for the example of static building!

I tend to use overlays to bring in the components. This allows greater control over variations and a coherent nixpkgs. I need to clean it up a bit more, but talked about it here:

https://www.reddit.com/r/NixOS/comments/8tkllx/standard_project_structure/

I like having the ability to call “nix build” and “nix-shell” directly in the respective sub-projects, but I do lose the niceness of having the same default.nix that nixpkgs expects, instead it is derivation.nix. And it is used by overlay.nix, release.nix, and default.nix for their various responsibilities.

I’ve been considering putting some proper thought into presenting a “best practice” project structure. Would you want to iterate on an approach and try to get it included in the nixpkgs manual?

@tom this is awesome, thanks! i will incorporate this, i really like the overlay approach!