I’ve got an available weekend to do some hours.
I’m also interested in hacking on an alternate store backend, Hydra, or content-addressability if anyone wants to pair program or just hang out.
I’ve got an available weekend to do some hours.
I’m also interested in hacking on an alternate store backend, Hydra, or content-addressability if anyone wants to pair program or just hang out.
“alternate store backend”, ooh, I’m a bit intrigued, can you say more?
Thanks for this great initiative, I’m hoping to have an interaction about nix and gain some knowledge from experienced folks. Sent an invite with more details.
I want to explore modern filesystems on top of the store. Things like using ZFS dedup, rsync transfer mechanisms, casync as a new NAR for better attribute/permissions management.
Thanks tom for inviting me to the session.
Here’s a summary of what we discussed last night.
Our initial discussion revolved around building developer environments for web applications with dependencies on stateful services typically a database and a reverse web proxy with the constraint that the environment has to be portable across linux distributions (not be NixOS specific).
We explored nix specific tooling for service management like nix-processmgmt, manual bash scripts and ways to turn nixos modules into containers/systemd scripts that can be run with different hosts.
We also explored procfile based solutions like foreman (and it’s ports like honcho, forego).
Container based service management options were also discussed notably arion and using nix to build docker images.
We then discussed building web applications written in dynamic scripting languages specifically php using nix with the goal to make local and ci builds the same.
Some of the tools we discussed in this space were:
We also discussed caching builds with private caches using nix copy
commands and using different binary substitutors.
I also gained the info that the official binary cache at cache.nixos.org
is never garbage collected, so it is possible to use old nixpkg revisions and rely on the cache being used for building packages.
Other topics we discussed:
This is very interesting, were there any conclusions on this topic?
Hi @tomberek
I could use some help from someone willing to donate some of their time to explain some python
issues with kodi
I’m having. If you’re willing to spend an office hours with me I would greatly appreciate it.
Thanks kindly!
Hi! I’d really want to either do something about a binary cache for pytorch-like stuff Add NVIDIA licenses by tbenst · Pull Request #76233 · NixOS/nixpkgs · GitHub (and I’m willing to invest my time in this) or just discuss how to currently evade the dlopen()
perils in a way that Nix doesn’t become an obstacle to work instead of help. Do you think it’d make sense for me to reserve a time slot?
We discussed a few solutions, as is typical in the nix world there are lots of approaches for doing similar things.
The idea that I got was there’s no single tool that can fit every requirement in portable dev envs space.
So there’s no clear conclusion right now, each application will have to pick the right solution for itself.
Right now I’m leaning towards using nix + docker by managing docker commands from within nix but I also like the idea of manual scripts that handle statefull services in an isolated manner.
@SergeK: yes, there is a nix-data group and cache focused on pytorch/ML/data libraries. For now it’s a loose collaboration of people that have been trying to maintain a Hydra and cache for that ecosystem. Feel free to reserve some time (during this week would be ideal, i’m out of town afterward.)
@aanderse: what kind of issues are you having?
I am thinking that there may be a path forward for a larger “office hours” effort. I’ve enjoyed this style of collaboration, but it does not scale alone. Providing some time in this fashion is different than general help in forums/chat - I believe is it of high-value and I’ve tried to focus on newcomers to ensure we have a welcoming community, but also takes time. There is also the consideration that there are existing consultancies that would benefit from helping Nix customers for packaging and training.
@grahamc: You were mentioned earlier as having done a regular online video meeting, how did that go? Thoughts/comments?
My issue has resolved itself, but I still don’t understand why. Some questions regarding PYTHONPATH
and kodi
and how they work together.
Office Hours with @grahamc and @worldofpeace were posted on the NixOS YouTube channel so you can check them out. Basically the hosts would discuss a topic, interview someone, then conclude with a PR review. Almost talk show style. They were fun, but very different from what you do. Yours are very hands on and dig down to the root of a problem.
Just got done with some vacation and I can do some more hours this week if people would like. Please let any newcomers or Nix beginners know.
I’d like to thank @tomberek for answering a lot of questions from me during an office hours session he hosted today.
We talked about the relationship between channels and pinning and flakes, and how to structure a system composed of the public nixpkgs repo along with privately-maintained packages. We also covered how to patch closed-source libraries, and some options for how to distribute Nix applications to hosts that do not have Internet access. It was super helpful to me.
@tomberek also asked me questions, and in particular wanted to know what I thought could be improved in the docs.
I pointed Tom to @ianthehenry 's excellent series of blog posts, where he reads the entirety of the Nix docs, and writes about what he understands and what he doesn’t understand after reaching each section.
As a Nix beginner, I really enjoyed seeing another soul struggle to make sense of the Nix world, usually in the same way I was, and for the same reasons. But I also hope Ian’s series is used to improve the Nix docs.
Office hours are open on a regular basis from now on. Feel free to schedule at Calendly - Thomas Bereknyei
I want to thank Tom for letting me have this awesome OfficeHours experience.
Coming from ops background and having little knowledge of languages, especially functional programming, I’ve been struggling to get past this learning curve of nix.
Still am, however I am very happy that in this short hour we managed to go through:
builtins.trace
I wish I knew at the time I picked up learning nix.Honestly as I saw Tom explaining I wanted to stop him at every other sentence and try to expand on things. I’ve got enough homework to work on.
Thank you so much for your time and this initiative! I did miss 2 hours of my OfficeHours that I had to catch up on, therefore I was unable to checkout General Nix Office Hours on jitsy but I hope there will be more of those in the future!
yes! thx, was googling for distrology
Had a great session.
I specifically had a question about a situation I got myself into where I had about 60GB of duplicate packages. I found out where most of those duplications came from before the call (I used the -p
option on nixos-rebuild
wrong), but some packages were still duplicated multiple times (apparently).
Tom told me about the following tools to explore where these duplications came from:
nix-store --delete /nix/store/foo
gives reasons why a package cannot be deleted.nix-store --query --referrers /nix/store/foo
shows which packages refer to the given package.nix why-depends /run/current-system /nix/store/foo
recursively lists packages requiring the specified package, basically a more powerful version of nix-store --query --referrers
.nix-store --gc --print-roots | grep -v proc
prints gcroots which are not a result of running processes.After doing this for several seemingly duplicate packages we found that they were not actually duplicate but instead came from packages requiring different versions of the same library.
Tom also recommended me the great nix-du
tool which uses graphviz to visualize dependency graphs of nixos installations based on size.