Tom and I just talked for a few hours about a problem I’ve been trying to overcome - cross compilation with darwin as the target. We didn’t get to a solution, but we did explore several avenues that look promising, including virtualization and remote builders.
He also gave me a few names of other Nixers who might be able to help more.
We also talked about some ideas and projects we’ve both been working on involving nix as a build system (rather than a package manager), and he explained Nomia to me so I finally understand what it’s about.
Thanks for this great initiative, I’m hoping to have an interaction about nix and gain some knowledge from experienced folks. Sent an invite with more details.
I want to explore modern filesystems on top of the store. Things like using ZFS dedup, rsync transfer mechanisms, casync as a new NAR for better attribute/permissions management.
Our initial discussion revolved around building developer environments for web applications with dependencies on stateful services typically a database and a reverse web proxy with the constraint that the environment has to be portable across linux distributions (not be NixOS specific).
We explored nix specific tooling for service management like nix-processmgmt, manual bash scripts and ways to turn nixos modules into containers/systemd scripts that can be run with different hosts.
We also explored procfile based solutions like foreman (and it’s ports like honcho, forego).
Container based service management options were also discussed notably arion and using nix to build docker images.
We then discussed building web applications written in dynamic scripting languages specifically php using nix with the goal to make local and ci builds the same.
Some of the tools we discussed in this space were:
We also discussed caching builds with private caches using nix copy commands and using different binary substitutors.
I also gained the info that the official binary cache at cache.nixos.org is never garbage collected, so it is possible to use old nixpkg revisions and rely on the cache being used for building packages.
Other topics we discussed:
Nix content based address derivations and their usefulness for caching builds.
Doing integrations tests within nix and how to test multiple package versions (passthru tests).
Securely verifying build outputs from different builders, example tweag/trustix.
I could use some help from someone willing to donate some of their time to explain some python issues with kodi I’m having. If you’re willing to spend an office hours with me I would greatly appreciate it.
Hi! I’d really want to either do something about a binary cache for pytorch-like stuff Add NVIDIA licenses by tbenst · Pull Request #76233 · NixOS/nixpkgs · GitHub (and I’m willing to invest my time in this) or just discuss how to currently evade the dlopen() perils in a way that Nix doesn’t become an obstacle to work instead of help. Do you think it’d make sense for me to reserve a time slot?
We discussed a few solutions, as is typical in the nix world there are lots of approaches for doing similar things.
The idea that I got was there’s no single tool that can fit every requirement in portable dev envs space.
So there’s no clear conclusion right now, each application will have to pick the right solution for itself.
Right now I’m leaning towards using nix + docker by managing docker commands from within nix but I also like the idea of manual scripts that handle statefull services in an isolated manner.
@SergeK: yes, there is a nix-data group and cache focused on pytorch/ML/data libraries. For now it’s a loose collaboration of people that have been trying to maintain a Hydra and cache for that ecosystem. Feel free to reserve some time (during this week would be ideal, i’m out of town afterward.)
I am thinking that there may be a path forward for a larger “office hours” effort. I’ve enjoyed this style of collaboration, but it does not scale alone. Providing some time in this fashion is different than general help in forums/chat - I believe is it of high-value and I’ve tried to focus on newcomers to ensure we have a welcoming community, but also takes time. There is also the consideration that there are existing consultancies that would benefit from helping Nix customers for packaging and training.
@grahamc: You were mentioned earlier as having done a regular online video meeting, how did that go? Thoughts/comments?
My issue has resolved itself, but I still don’t understand why. Some questions regarding PYTHONPATH and kodi and how they work together.
Office Hours with @grahamc and @worldofpeace were posted on the NixOS YouTube channel so you can check them out. Basically the hosts would discuss a topic, interview someone, then conclude with a PR review. Almost talk show style. They were fun, but very different from what you do. Yours are very hands on and dig down to the root of a problem.
I’d like to thank @tomberek for answering a lot of questions from me during an office hours session he hosted today.
We talked about the relationship between channels and pinning and flakes, and how to structure a system composed of the public nixpkgs repo along with privately-maintained packages. We also covered how to patch closed-source libraries, and some options for how to distribute Nix applications to hosts that do not have Internet access. It was super helpful to me.
@tomberek also asked me questions, and in particular wanted to know what I thought could be improved in the docs.
I pointed Tom to @ianthehenry 's excellent series of blog posts, where he reads the entirety of the Nix docs, and writes about what he understands and what he doesn’t understand after reaching each section.
As a Nix beginner, I really enjoyed seeing another soul struggle to make sense of the Nix world, usually in the same way I was, and for the same reasons. But I also hope Ian’s series is used to improve the Nix docs.
