Google Summer of Code?

I’m a college student and moderately-experienced Nixpkgs contributor looking at possible doing Google’s Summer of Code this summer, and I’d love to work on something Nix-related. It looks like Nix didn’t get registered as a mentor organization this year, but the Linux Foundation seems willing to mentor on behalf of organizations that missed the deadline, so I figured I’d bring this up and ask if there’s interest.

Below are some various ideas I’ve been interested in working on. I’m also very open to ideas not listed here!

  • Improving the NixOS installation/new-user experience
    • Putting together a graphical installer, maybe based on Calamares?
    • Getting nixpart up and running and merged into nixpkgs?
    • Building a way to programmatically read and modify NixOS configuration, for use by an installer or GUI configuration tools?
  • Improving language packaging
  • Experimenting with alternative toolchains
    • Building Linux Nixpkgs/NixOS on Clang, and experimenting with the resulting optimization opportunities (ThinLTO?)
    • Experimenting with a rust-based Coreutils?
    • Using nix-ccache in nixpkgs (this depends on Recursive Nix; not sure if that’s stable enough to start building on top of)?
  • Gathering and centrally collecting statistics from Nix builds (time? memory usage? disk usage?) to provide better time estimates and warning if there are insufficient resources for a build?
  • Running nix-packaged software on AWS Lambda (where, AIUI, we don’t have root and therefore we’d need some sort of chroot or container to write to /nix), and using Lambda as an infinitely-scalable remote builder?

I’d also love to hear from any other GSoC mentoring organizations interested in Nix-related work (or, for that matter, any companies using Nix looking for a summer intern).

13 Likes

@Gaelan For your last point, nix on Lambda, there a minimal PoC here:

The initial impetus was to use this with Stanford’s gg. It’s all ugly and a massive hack and is probably out of date by now, but it may help you in some way.

For alternative toolchains, i’m sure the ALLVM [ALLVM Research Project | LLVM All the Things! - University of Illinois at Urbana-Champaign] effort would love to make progress.

Hey, I’m no mentor, but I just want to say that your alternative toolchain ideas sound really interesting. If you do decide to take that route, feel free to ping me – I’d be happy to review PRs and help out where I can.

1 Like

I encourage you to contribute to nix, it would be awesome!
Around nix and nodejs, there was recently a discussion about how the how current nodejs packages doesn’t scale and is broken.
There was a discussion here

The need for a new solution was mentioned but no proper direction got put forward. I would personally say there is a big need for this. The current state of things is that it’s basically very hard to contribute any nodejs package.
I would be willing to review PRs and get the right people involved, I’m nowhere near the level of being able to provide mentorship though.

Check out GitHub - divnix/devos: NixOS Framework which seems very promising foundations.

ref: GitHub - DavHau/nix-portable: Nix - Static, Permissionless, Installation-free, Pre-configured

More discussion: Nix Portable: Nix - Static, Permissionless, Install-free, Pre-configured

Since it’s Google, it might be a fitting cabdidate to revamp k8s support from a nix perspective. Projects like GitHub - xtruder/kubenix: Kubernetes resource builder using nix lay a good foundation. That would also play well the other way round, e.g. a kustomize plugin to render from nix-lang / kubenix config.

1 Like

I have another suggestion for you which is in my opinion a bit more urgent than trying new things:

The way we package nodePackages needs a rework. The giant nodePackages file is not reviewable and node2nix stack overflows with some bigger packages like zigbee2mqtt (see zigbee2mqtt: 1.16.2 -> 1.18.1 by mweinelt · Pull Request #114896 · NixOS/nixpkgs · GitHub). I think it would be a good idea to fix those issues before we build node packages from source.

3 Likes

That’s a great idea! There is already work done that someone has to continue.

That would also be great. Not sure how outdated it is.

1 Like

For an additional idea on how to package nodePackages. pnpm installs packages with a content addressable store GitHub - pnpm/pnpm: 📦🚀 Fast, disk space efficient package manager . Perhaps this can be leveraged?

4 Likes

There lock files are also not tiny (pnpm/pnpm-lock.yaml at 53324fb37262045d8b13d1ad282b71f0787609e9 · pnpm/pnpm · GitHub) and because most repos don’t have pnpm lock files we would need to create them for basically all packages. On the other hand they take care of a lot of things for us already like dependency resolution and strict dependency which we definitely want.

I think this solution will not be perfect but it could be far better than the current node2nix approach and someone else already did quite a few tasks for us and hopefully maintains them. This could potentially solve stack overflows like mq2tt currently has, simplify nodePackages and remove quite a lot of boiler code.

I am personally would be very interested in this project and if you (or someone else) wants to do it I would be open for questions and would like to share my (limited) knowledge about nixpkgs. Just mention me on IRC or write me a private message. Also if someone got a prototype working and wants to test it for nodePackages but lacks the hardware or bandwidth just create a draft PR or send me a link to a git repo with a short message and I should be able to build it within a few hours with the remote builders I have access to.

1 Like

Just to make sure I understand correctly.
The idea would be to install a package with pnpm then recusively convert the symlinks to the packages to the nix store.

The way to start would be to take a package with no dependency and try to convert the pnpm format to nix. Maybe something like function-bind (function-bind - npm) (I just did a quick search, anything would work). The second step would be to figure out the recursive part.

Just making sure I understand the concept. Feel free to correct me.

Yeah I think it’s crucially important than our language and language-specific package manager support be better, but until we find something like IFD to make using those in Nixpkgs productive, I think it’s a bit premature to have a GSOC on them :/. Right now the sad truth is Nixpkgs is our best community effort but the best-designed language-specific stuff proposers despite and not because that community energy :(.

1 Like

as a minor aside, there is already The Summer of Nix.

5 Likes

Ooh, I wasn’t aware of that! I’ll apply.