I meant targeted in the sense of people who know that automatic merging in nixpkgs exists and exploits that as a weakness.
Why not do it slightly more carefully and have a chance to attack everyone who is not deliberately one year behind?
Of course, you have to get some people to install the software. But you could introduce a package in nixpkgs without any malicious code and then later get a flavor merged that does whatever it needs to do. Since this would only affect nixpkgs,
(this would be false specifically for r-ryantm PRs, but I agree we might want wider auto-merge eventually if the first stage goes well)
it wouldn’t be caught by someone else and covered in any security news.
If it is a package worth any attacking effort, the infrastructure breach itself might be newsworthy.
So I probably should have fleshed this out more. r-ryantm’s PR’s are generally one or two lines that increment a version number and a hash (or maybe 2). I think restricting it to patches which match that (via an auto merger that checks that), would likely reduce a lot of attack vectors.
In fact one of my side projects is to write an auto merger like this and just run on top of a bunch of auto merged PRs to see how viable it is.
move failure detection into nix
Yeah - One of the things on my list for go before I got jammed in code review misery was to flip the default to run the unit tests in go - that should catch lots of issues before they happen + provide much better confidence.
In general, I wonder if we could even provide a warning when a package doesn’t have tests that it needs extra review + is likely a low quality package.
(writing this on phone…)
FWIW there is (was?) an RFC for implementing SELinux and in my opinion this (or alternatively apparmor) could already add some „security“. It‘s been standard for Redhat derivates for a long time and now also for Debian (idk about others). People are always scared of SELinux/Apparmor but it‘s really not that big a deal once you get used to it.
As a package maintainer for two (niche) leaf packages I would be very much in favor of a) some kind of metadata telling ofborg „how much security is already in the package“ and how much human interaction is expected. The topic of splitting packages into core/community has existed for a long time and instead adding a metadata field would in my opinion make more sense for finer grained definition. Of course a lot of things would have to be considered like changing metadatafields should not be allowed to be merged automatically.
This thread is very inspiring to me as I have never thought of looking at release diffs of packages (bc I just „trusted“ the developer - not very safe!!) or I’ve never really seen custom tests in NixOS for leaf packages (and don‘t know how that works exactly). I for one would be VERY glad if this thread at least leads to someone (de docs team?) writing up a wiki article/appendix to the manual explaining good behaviour when maintaining a package. this could be combined with the mentioned metadata field. E.g. „the metadatafield can be set to ‚community - manual review’ if your packaging process is following these standards:
You manually review the diff of the last release
Package tests are run (manually or automatically)
…
This is a very bad example but I guess you get the idea.
Try to find librealsense in guix. I tried adding Python support. Worked. Upgrading 2 versions → cmake fails to find Python again (WTF). This is what is happening all the time.
But there are worse packages such as 3dslicer.
You try to package it. Then you’ll find it requires many downloads. You hack cmake to allow passing downloads via env vars … then you just notice it turns worse cause it check outs from git several packages itself and even writes tests for it or such.
What I want to say both guix and nix are damn cool. No doubt. But they still fail because you need to to combine 1) version control 2) building 3) dev workflows 4) tracking of reviews by others people of changes etc in one system. If anybody wants to work on this let me know cause I’ve been running into this kind of issue way to often. And I am tired of having to hack much myself. We have 2020 and things should just work or break for dev reasons not packaging reasons anymore. By packaging I don’t mean how nix works. I mean how people use ‘cmake’ like in the case above due to lack of alternatives.And going fast (breaking things) is also costly.
Then next: Hey, yeah rust works (not latest version, but … hey)… rust realsense → compilles → hey… but the downloaded rust package requires different realsense.so despite having asked for nix realsense headers (WTF)…
All I want to say is that the next step is not about guix vs nix… The question is how to write a github which allows coding and compiling and package management and downloading things on the fly as needed IMHO. Nix has done an incredible job raising the bar. But packaging some packages is still too much pain.
But the journey continues:
Packaging JS is fast. type checking TS is much slower.
Yet there are many proofs you cannot ensure with TS (such as SQL injection attacks whatever).
So what you really need is also a language going beyond:
fast compilation to make it work
second phase type checking to check whatever you want
But writing this kind of tooling is going to be a 2 year+ journey + libraries
BTW there are more projects which suffered from ‘leadership’ in some regard, yet many people still use it (but suffered). Such as Vim. Eg package managers for vim were written multiple times because Wiki and and plugin systems were separate and maintained by different people.
¿How can we consolidate momentum? (and get to action - just as so to fully and exhaustively convince @doronbehar we don’t need a carismatic leader - Max Weber style)
@danieldk I appreciate how you steadily complement most of the “Strum & Drang” inputs with important second thoughts. May I ask you to try to transact some of those inputs into small and actionable postulates with low pareto-inefficiency (just low enough so nobody would dare to veto)?
Things I’m recollecting from this thread:
Strengthening of the RFC routines (evtl. Lifecycle Bot - Now? Later?)
Improve Segmentation of Decision Making to some prudent extent like per language ecosystem or similar (aside: some of the Germans here do surely know what they owe to the concept of Federalism)
Strategically improve PR automation and workflows (through MVPs + iteration and shared conceptualizations)
For my part, I’d like to be able to deduce from such consolidated “call to action” how to shape and conceive aspects of my current and future contributions.
Has a ok-to-test stage - our human intuition - but before taxing testing infrastructure too much.
EDIT: it removes comments which are acted upon
EDIT: uses traffic light label color codes for immediate signalling
Klicking a little thorugh this bot’s background, I found this for automatic mergin in k8s: Tide Status (It’s based on gihub search queries based on labels - elegant!)
However as the RFC process suggest, I’d invite interested party to share their feedback and thoughts on my private fork’s PR previously to submitting a PR to NixOS/rfcs. Thanks in advance!!
We’re looking forward to the feedback to come! If you wish, feel free to be creative in the Brainstorming section. The section’s purpose is to stimulate imagination and get you on board!
Considering my original question of this thread. Now I must say it feels great to contribute in this kind of period - where we are close to a scheduled release and we have a release manager (@jonringer & @worldofpeace). You are a Benevolent dictator, but not for life so perhaps this setup is perfect for NixOS .
The next step is that i document it and then we can encourage contributors to add tests. But i do many things and have limited time… feel free to help!
I have daydreamed a little about whether it’d be helpful to have a system for nudging package users about things they use with a low-friction way to collect feedback, and a parallel project to find UI/X wins that leverage that feedback.
I think I’ve written about this at more length before, but some meaningful things in that category might be:
collect meaningful test cases from people at (or near) the point/time of use
notify people when something from nixpkgs in their system/user config or a nix-shell doesn’t have a maintainer and help them adopt it
nudge people to mark themselves as known-users (would need some GC mechanism?) that a maintainer can ping for feedback/information
report build failures automated transparent collection of relevant platform information, and maybe some light troubleshooting (“It looks like you have sandboxing configured; does it build if you add --option sandbox false?”)
likewise, report this-is-working-to-the-best-of-my-knowledge signals that may help with triaging or even self-service help with some classes of problem (i.e., providing a way for a maintainer or even end user to quickly see if anyone’s reported this version as working on the same platform; “I noticed failed to build; you can run nix hobgoblin <store-path> to see if anyone has had better luck with it.” or even, “Fair warning: no one on your platform has reported successfully building this, yet. Is it okay for me to automatically report whether it worked or not?”)
Due to Cachix funding done in open source spirit (also known as bootstrapping funding) and due to my personal savings I’m able to spend 100% of my work time on Nix and Cachix.
My mission is to bring Nix ecosystem to en masse.
Each customer using Cachix contributes to making that sustainable, mostly due to ease of setup and not relying on never GC-ing your cache and using Cachix as a real non-permanent cache. Lots of small teams appreciate that!
I believe the major obstacles lie within documentation and infrastructure, something I’m focusing on for the last two years and will continue doing that for years to come.
Would it make sense to make a generic test where we could specify how to get the version (like with --version and maybe a regex) to avoid having to put a test.nix file with every package?
I know myself and many others appreciate your work.
One of the big blockers to using nix at my work (besides bus factor) is the “risk” concerns around how supported it is. Products like Cachix and integration into other ecosystems like github actions helps demonstrate that it’s a valuable platform.
Being able to easily “show” value, goes a long way to adoption.
As an outsider and out of curiosity, is there a way to profit in some way of the security effort that other distro’s maintainers do while they review the same sourcecode for their distro that in the end is packed for nix, too?
It is like waiting a week or two (waiting if someone else finds security issues), but actively communicating about what others find.
Maybe this would end up in a lot of overhead, but from a security point of view, knowing how many eyes (maybe also who’s eyes) have reviewed code is a good way to estimate the risk of malicious code not being discovered.
It would be like: ok maintainer of Debian, Arch, Fedora didn’t find anything, so it should be save to package… This way, the effort between the distros would be shared.
I don’t think there’s any official platform that type of discussion between package repositories.
But generally if there is an issue with a package, then most nixpkgs contributors will let the upstream source know about it. However, this is usually related to packaging.
From a code review perspective, I can’t speak for others; but I do very little code review unless there’s regression or behavior change in a program. I will usually review the changelog if it’s a popular package (or causes many rebuilds).
Limited knowledge but I don’t know if package maintainers review application code itself in any distros as a matter of general process. Most code they review is the build related instructions or additional patches for old software not found working.
I like the idea of having some ‘trust indicator’ for automerging new upstream versions.
Example
Nixpkgs contains foo-3.4.1
Upstream project ‘foo’ releases foo-3.4.2.
To minimize human intervention to merge this into nixpkgs, let the bot check if (e.g.) Debian sid also has this version. If Debian does, then this is one of the (multiple) factors that could lead to an auto merge.
- our human intuition - but before taxing testing infrastructure too much.
.