I love the concept of NixOS, but the amount of broken packages in Nix is making me think 80,000+ packages in a repo is a bit of an exaggeration…
I was using a popular package today: FreeFileSync. While FreeFileSync works fine, but there are some features that simply don’t work:
I can’t open settings or any configuration settings
I can’t open Cloud server option
Other popular or useful packages that are broken or bad are:
Ardour
Audacity
BTRFS Assistant
Freac
It sucks when this happens. I usually default to Flatpak to save the day, and it’s been a great crutch. I have nothing against Flatpak, but ideally, I’d be nice if things would work on Nix. If packages are broken, they should not be available for people to include into their configuration.nix file.
When packages are broken, how do we report them? Is there anything we can do about it?
Speaking very generally, I’ve never used a software repository that I didn’t find some broken software in. Sometimes it’s the software’s fault, sometimes it’s the packager’s fault, and sometimes it was fine when it was packaged and has rotted a bit as its dependencies get updated but the package itself doesn’t evolve.
Nixpkgs’s infrastructure and practices are fairly good about keeping complete breaks from creeping in, but spotty/partial breaks are hard to ward off. All we really have there is a community of engaged users who take the time to mend things when they find them broken.
If you think you have a defensible case that it’s sufficiently broken to be unfit for any use whatsoever, it would probably be fine to open a PR that sets meta.broken with a code comment explaining why and links to the issue.
Figure out why it’s broken, how to fix it, and open a PR doing so.
I’m not. In fact, I’m fine if none of these packages ever gets fixed. I know most of this work is voluntary, and I’m in no rush to see any of these things work. Fortunately, in the meantime, I can use flatpak.
But I would just prefer not seeing a broken package listed at all. Or, it’d be nice if we could see some indication that the package doesn’t work.
When I configured my config.nix file, I would search packages that I wanted to add to my system. If I had known that these packages wouldn’t work, I wouldn’t have gone through all that effort to compile a list.
I understand that fixing these broken packages is up to the maintainer or packager. I don’t understand how these broken packages get pushed into release.
I think what would help in filtering good and bad packages is seeing peoples’ negative reviews and comments of a package not working, or maybe seeing flags that show that the package doesn’t work.
“broken” is a bit of a spectrum, though. For some usecases, it might not matter that some functionality is not working. Of course, if the software is completely unusable, it is objectively broken and should be marked as such.
Yeah, that’s a bit of an issue generally, the metadata in nixpkgs leaves a bit to be desired. I think adding this to the description of the package is a decent solution. This way, the breakage is discoverable by users both through the CLI search tools and search.nixos.org, without preventing anyone who doesn’t care about the limitations from installing the package anyway.
You could try to open a PR that adds these sections to the description of the packages you mentioned. I think that would be a relatively uncontroversial change. Maybe you could even include a link to the github issue, though I’m not sure if those would be rendered by all tools properly.
Of course a general solution for this would be nice. Maybe a meta.bugs or meta.known-issues field with a list of links?
There was an RFC for usage notes at one point, iirc, suggested because a lot of packages should be installed via their .enable flag, but this is not visible in package search, so often - especially newcomers - end up installing packages in a broken state. Wouldn’t surprise me if this applied to some of the packages for this thread too
That could also be used for this, after all it’s part of usage if the packages aren’t outright broken. I wonder what happened to that discussion.
I remember when people were excited by Google’s page rank algorithm (searchworthyness determined by count of inbound links). It’s clearly not perfect, but it has some merit. Maybe we could do something like that.
References to nix packages show up in places like github–either as a “my nix config” repo, or in project repos for use in a nix shell or somewhat. I don’t know about you, but if I fail to make the thing work, I remove it from my code.
So presumably a high count of references in an active repo is an indicator of the package being useful to someone. If we wanted to get real fancy we could look at git history and identify cases where the package was added and later removed in a short period of time (a negative indicator), or have users indicate trust in each other such that references authored by people they trust (maybe transitively) count for more.
From there, you just ignore everything below a certain threshold.
I’m shooting for rank and not removal because I think that discovering a broken package and moving on is a small annoyance compared with not being able to share your package with your friend because the elders refused it. We don’t want to scare people off who will grow up to be competent package maintainers.
Counting references in github might be possible, but could be expensive to query as well, and I wonder how reliable the results of such an action would be.
“Quality” is a metric composed of checking whether the project has things like a readme and contributing.md. “Maintenance” checks the github repo for open issues and update frequency.
Neither of which necessarily implies anything of value, but I guess there will be some correlation with perceived project maturity.
Number of open/closed issues regarding a package could be an interesting metric, or - as a feature of search.nixos.org - linking to a filtered list of issues regarding the package. I guess npm rebrands it as “maintenance” since #issues might seem a bit negative, when the take-away should probably be that actually it has a lot of attention.
All of this would take work gathering metadata that to my knowledge currently is not recorded at all, though package downloads was at least planned at one point.
It seems like pulling metrics directly from the binary cache would be a good indicator of usage rather than scanning GitHub (which contains private repos you can’t scan).
I agree that github wouldn’t make this easy. I often try–and fail–to use their advanced search for queries like:
show me popular repositories with both a flake.nix and a .pre-commit-config.yaml in them
They just don’t have the repo’s indexed by content in the right way to make this kind of analysis possible (or if they do, they’re failing to make it easy to leverage).
I donned my tin-foil-hat and gave this a second thought. We should be careful to avoid creating a situation that’s too easy to game. 98% of package maintainers wouldn’t go out of their way to abuse a ranking system. The remainder is two groups:
1% attention-seeking–and I suppose we can forgive them for that supposing their software is good
1% trying to get you to run their malware
Not that it has to be airtight–all attempts at metrics of this sort are flawed–but if we should take care that whatever we build does not make life significantly easier for the second group.
The status quo, while inconvenient for the good guy, is at least not overly convenient for the bad guy. They can’t just leave a script running which generates noise that elevates their package over others.
