2023-06-19 Tutorial Series Call for Feedback #1

The Learning Journey Working Group is starting a work process in which we focus on a small number of tutorials for a month so that we can get them into a state that contributors can pick them up and work on them as soon as possible.

Part of that process is acknowledging that the small number of people in the working group, who are mostly Nix beginners wanting to make it easier for other beginners to learn Nix, don’t know all the ins and outs of the Nix ecosystem or where the pitfalls are.

We’d like to leverage the community of experts and anyone else willing to pitch in.

Going forward we’ll solicit feedback and suggestions each month for the tutorials we’ve selected for development.

This is the first time we’re trying this so I’m sure there will be some trial and error and growing pains.

How you can help

Suggest activities

For each of these tutorials we’d like the user to walk through some activity that will put them into contact with the topic of the tutorial.

We’ll have some ideas of activities users can walk through on their own, but more suggestions at this stage of the process will always be better.

So, the first thing you can do is provide suggestions for activities that users can do on a particular topic.

Where do users get stuck?

The next thing you can do is provide us with insight into where users get stuck on these topics.

Even if all of us in the Working Group were experts, we don’t know everyone using Nix, and we certainly don’t know where everyone gets stuck.

If you have experience helping users with these topics, please point out the major pain points or stumbling blocks.

Make us aware of the traps

Anyone who has ever used a computer has had the “how hard could it be” experience that turns into a rabbit hole of complexity.

If you know that a particular topic we’ve chosen has hidden complexities, we’d like to hear about them so that we can keep the reader on a happy path until we intentionally decide to present any complexities.

For example, if you bring python310 and python310Packages.pip into a nix-shell -p environment, you can’t just install Python packages with pip install like you would outside of a Nix shell.

The tutorial ideas

Feel free to leave comments either on the GitHub issues linked below or in this thread.

Sharing dependencies between default.nix and shell.nix

GitHub Issue

The idea here is that we’d like to demonstrate how to set up a development environment, which means bringing packages into the shell other than those that are strictly part of the build.

One idea here is to take a project with a working default.nix that simply builds the project, duplicate the dependencies to shell.nix, then show how you can share the dependencies between shell.nix and default.nix via the inputsFrom attribute passed to mkShell.

Questions:

  • What kind of project should the user build?

  • Which other tools should the user bring into the shell? Examples could be linters, formatters, etc.

  • Anything we should watch out for?

  • Any best practices in this regard?

Packaging existing software

GitHub Issue

This is a big topic, but what we want to teach in this tutorial is the workflow, not necessarily every possible way to package existing software.

It’s common to have be missing dependencies that you didn’t know that you had, so we want to show users how to figure which ones they’re missing, where to find the packages they’re looking for, etc.

Also, it’s common for header files to be under a mypkg.dev attribute, so we’d like to make users aware of that as well.

One idea here is to take a working project that may have language-level dependencies (e.g. Rust crates, Python modules, etc), but also some system dependencies.

For example, if you want to use a library that wraps libcurl you’ll need to bring libcurl into your build even though you might only directly depend on a Python module called easy_curl (made up arbitrary package name).

Questions:

  • What kind of project should the user work on that would allow them to focus on the Nix aspects and not on the particularities of the language?

  • Are there common workflows you use to track down why your build is failing?

  • Can you think of a project that requires a variety of different methods to get it working e.g. requires more than “I’m missing foo, add package foo”?

Trivial builders

GitHub Issue

Sometimes you don’t need the full Nix machinery of building and compiling an application.

Sometimes it’s fine to run a few commands and just make sure something is in $out by the time you’re done.

This is the use case for trivial builders that we’d like to demonstrate.

One idea is to build up a shell application starting from writeTextFile and manually put it in a /bin directory and make it executable, then show how you can accomplish the same task by using writeScriptBin or writeShellApplication.

Some really valuable feedback here would be use cases where you’ve done this yourself to solve a problem or accomplish a specific goal e.g. what can we show the user here that would be useful?

mkDerivation phases

GitHub Issue

Derivations can be broken down into various phases, and if you know the contract for each phase or when they run relative to one another you can better mold a derivation to suit your needs.

What we’d like to show the user here is how to use some of the more commonly overridden phases to solve their problems.

One idea could be to have the user build their own builder for a particular language, showing how the builder in nixpkgs implements each of the phases to handle the particularities of that language’s build process.

One pitfall in that example is that it could lead the user to focus too much on the language and less on the Nix aspects.

Questions:

  • Which phases would you focus on?

  • What are some use cases for specifying the phases manually?

10 Likes

Regarding packaging, I am personally getting somewhat comfortable with nix (still a noob), but I still have no clue how nixpkgs and all helper functions and patterns work. I am not sure, if this is within scope of the tutorial, but getting into the peculiarities of nixpkgs would be helpful at least to me at some point.

1 Like

Love your provides ideas.

Things that I struggled or am still struggling with:

  • Nix language, I’ve never really had a formal introduction to a functional language and still don’t really get how Nix works. Maybe something like this is how you do X in Python this is how you do it in Nix.
  • How to properly package something like a Django application and maybe put into a container as well? At lot of what I’ve seen is building a binary and not building a website or a service.
  • Explain the basics of the dynamically and static linked libraries. I’ve never coded in C or C++ before so I know this concept only very roughly.

I’ve been using Linux for quite a few years and currently as a systems engineer/devops guy but I very rarely had to build something myself on another distro (mostly Debian or Arch based) so a lot of the problems in NixOS never really happened or when they happened they were often solved with “install this package”.
I’m not saying that one approach is better than the other (clearly NixOS is better but don’t tell anyone), but I reckon for a lot of people having to build packages and understanding the tooling around might be quite new.

Could you provide some examples of situations in which you were confused, didn’t know where to go next, etc? That would help us focus our efforts. Also, many of us working on this are also noobs and may not have run into the weirdest stuff yet.

Does the Nix language basics fit the bill for what you’re looking for regarding the language?

Packaging a service is something we could cover, although a website or a service is really just another binary (e.g. a Django app would be packaged with buildPythonApplication). Are you more interested in how to configure NixOS to connect multiple services together e.g. have systems automatically start your app when the machine starts?

For example:

  • How overrides/overlays etc work, there seem to be multiple ways/conventions (or at least there were) to change/adapt existing packages
  • How to add a package/module to nixpkgs, where to put files, calling conventions (for example, I only recently figured out, that callPackage just throws all of pkgs, including pkgs itself at the called function, which then lazily picks what it wants), structure, what helper functions exists, how are they used and defined
  • How to add a package locally in your config in NixOS without having to create a PR in the nixpkg repo
1 Like

Not really.
A lot of the tutorials show just the basics and a hello world but at least for me that is not enough to understand a lot of the code that is out there.
A tour of Nix goes into the right direction but it could do with a a few real world examples.
I’m currently reading “An Introduction to Programming in Emacs Lisp” and I really like how it guides you through real code and explains what it does.

Yeah probably.
Like I have it packaged, how do I combine it to a proper service.
Or just stuff like how do you use composer or poetry when packaging?
Do we have to package every dependency or is there a middle way?
E.g. is it okay to use poetry install instead of using poetry2nix?
How do we deal with static files?

Writing System Units is actually something that I think is easier with NixOS than with other distros because I don’t have to move the files around.
I just write the service/timer and Nix takes care that everything ends up where it has to be.

What I found relatively off putting is that Nix does not accept my project local dependencies but needs to nixify the entire world. I ran into blockers with this for each of cargo, node and python.

Then you need to either use a function that understands your project structure or throw away half a day fighting with dream2nix.

In short I gave up and am using system global tool version managers (rustup, pyenv, etc.) to facilitate development on my machine.

I think this is an area that requires a lot more clarification.