Nix Office hours

Help
26

I’ve got an available weekend to do some hours.

I’m also interested in hacking on an alternate store backend, Hydra, or content-addressability if anyone wants to pair program or just hang out.

1 Like
27

“alternate store backend”, ooh, I’m a bit intrigued, can you say more?

28

Thanks for this great initiative, I’m hoping to have an interaction about nix and gain some knowledge from experienced folks. Sent an invite with more details.

29

I want to explore modern filesystems on top of the store. Things like using ZFS dedup, rsync transfer mechanisms, casync as a new NAR for better attribute/permissions management.

1 Like
30

Thanks tom for inviting me to the session.

Here’s a summary of what we discussed last night.

Our initial discussion revolved around building developer environments for web applications with dependencies on stateful services typically a database and a reverse web proxy with the constraint that the environment has to be portable across linux distributions (not be NixOS specific).

  • We explored nix specific tooling for service management like nix-processmgmt, manual bash scripts and ways to turn nixos modules into containers/systemd scripts that can be run with different hosts.

  • We also explored procfile based solutions like foreman (and it’s ports like honcho, forego).

  • Container based service management options were also discussed notably arion and using nix to build docker images.

We then discussed building web applications written in dynamic scripting languages specifically php using nix with the goal to make local and ci builds the same.

Some of the tools we discussed in this space were:

I also gained the info that the official binary cache at cache.nixos.org is never garbage collected, so it is possible to use old nixpkg revisions and rely on the cache being used for building packages.

Other topics we discussed:

  • Nix content based address derivations and their usefulness for caching builds.
  • Doing integrations tests within nix and how to test multiple package versions (passthru tests).
  • Securely verifying build outputs from different builders, example tweag/trustix.
4 Likes
31

This is very interesting, were there any conclusions on this topic?

32

Hi @tomberek :wave:

I could use some help from someone willing to donate some of their time to explain some python issues with kodi I’m having. If you’re willing to spend an office hours with me I would greatly appreciate it.

Thanks kindly!

33

Hi! I’d really want to either do something about a binary cache for pytorch-like stuff Add NVIDIA licenses by tbenst · Pull Request #76233 · NixOS/nixpkgs · GitHub (and I’m willing to invest my time in this) or just discuss how to currently evade the dlopen() perils in a way that Nix doesn’t become an obstacle to work instead of help. Do you think it’d make sense for me to reserve a time slot?

34

We discussed a few solutions, as is typical in the nix world there are lots of approaches for doing similar things.
The idea that I got was there’s no single tool that can fit every requirement in portable dev envs space.
So there’s no clear conclusion right now, each application will have to pick the right solution for itself.
Right now I’m leaning towards using nix + docker by managing docker commands from within nix but I also like the idea of manual scripts that handle statefull services in an isolated manner.

1 Like
35

@SergeK: yes, there is a nix-data group and cache focused on pytorch/ML/data libraries. For now it’s a loose collaboration of people that have been trying to maintain a Hydra and cache for that ecosystem. Feel free to reserve some time (during this week would be ideal, i’m out of town afterward.)

@aanderse: what kind of issues are you having?

I am thinking that there may be a path forward for a larger “office hours” effort. I’ve enjoyed this style of collaboration, but it does not scale alone. Providing some time in this fashion is different than general help in forums/chat - I believe is it of high-value and I’ve tried to focus on newcomers to ensure we have a welcoming community, but also takes time. There is also the consideration that there are existing consultancies that would benefit from helping Nix customers for packaging and training.

@grahamc: You were mentioned earlier as having done a regular online video meeting, how did that go? Thoughts/comments?

36

My issue has resolved itself, but I still don’t understand why. Some questions regarding PYTHONPATH and kodi and how they work together.

Office Hours with @grahamc and @worldofpeace were posted on the NixOS YouTube channel so you can check them out. Basically the hosts would discuss a topic, interview someone, then conclude with a PR review. Almost talk show style. They were fun, but very different from what you do. Yours are very hands on and dig down to the root of a problem.

1 Like
37

Just got done with some vacation and I can do some more hours this week if people would like. Please let any newcomers or Nix beginners know.

3 Likes
38

I’d like to thank @tomberek for answering a lot of questions from me during an office hours session he hosted today.

We talked about the relationship between channels and pinning and flakes, and how to structure a system composed of the public nixpkgs repo along with privately-maintained packages. We also covered how to patch closed-source libraries, and some options for how to distribute Nix applications to hosts that do not have Internet access. It was super helpful to me.

@tomberek also asked me questions, and in particular wanted to know what I thought could be improved in the docs.

I pointed Tom to @ianthehenry 's excellent series of blog posts, where he reads the entirety of the Nix docs, and writes about what he understands and what he doesn’t understand after reaching each section.

As a Nix beginner, I really enjoyed seeing another soul struggle to make sense of the Nix world, usually in the same way I was, and for the same reasons. But I also hope Ian’s series is used to improve the Nix docs.

7 Likes
39

Office hours are open on a regular basis from now on. Feel free to schedule at Calendly - Thomas Bereknyei

4 Likes
40

I also had an awesome experience with @tomberek during the office hours - thanks a ton!

41

I want to thank Tom for letting me have this awesome OfficeHours experience.
Coming from ops background and having little knowledge of languages, especially functional programming, I’ve been struggling to get past this learning curve of nix.
Still am, however I am very happy that in this short hour we managed to go through:

  • Talk about flakes. The problems it’s trying to solve. Benefits of using it.
  • Helped me better understand lib.mkOption. How to use it when creating variables. (And I thought I knew what I was doing.:sweat_smile:)
  • Talked about IaaS and tools(NixOps deploy-rs morph krops) available to manage it.
  • Learned about a builtins.trace I wish I knew at the time I picked up learning nix.
  • Learned about existing matrix community channel.
  • Presented some fancy stats about nix having 60+ packages and surpassing AUR. Lost the link.

Honestly as I saw Tom explaining I wanted to stop him at every other sentence and try to expand on things. I’ve got enough homework to work on.

Thank you so much for your time and this initiative! I did miss 2 hours of my OfficeHours that I had to catch up on, therefore I was unable to checkout General Nix Office Hours on jitsy but I hope there will be more of those in the future!

3 Likes
42

@simisimis I think this is probably the link you lost: Graphs - Repology

43

yes! thx, was googling for distrology :sweat_smile:

44

Had a great session.
I specifically had a question about a situation I got myself into where I had about 60GB of duplicate packages. I found out where most of those duplications came from before the call (I used the -p option on nixos-rebuild wrong), but some packages were still duplicated multiple times (apparently).

Tom told me about the following tools to explore where these duplications came from:

  • nix-store --delete /nix/store/foo gives reasons why a package cannot be deleted.
  • nix-store --query --referrers /nix/store/foo shows which packages refer to the given package.
  • nix why-depends /run/current-system /nix/store/foo recursively lists packages requiring the specified package, basically a more powerful version of nix-store --query --referrers.
  • nix-store --gc --print-roots | grep -v proc prints gcroots which are not a result of running processes.

After doing this for several seemingly duplicate packages we found that they were not actually duplicate but instead came from packages requiring different versions of the same library.

Tom also recommended me the great nix-du tool which uses graphviz to visualize dependency graphs of nixos installations based on size.

5 Likes
45

Thank you for the session, @tomberek . Learned about:

And more.