Hi, can anybody tell me where to find the feed of nixpkgs security issues in machine-readable format? The goal here is to write a script that checks my installed packages against the latest list of vulnerabilities.
The nixpkgs manual points to this github issue search query:
The issues do seem to have a subject-line naming convention, although it seems a bit risky to try to parse out the pname from the subject. And having to create a github account and use their API (and convince them it’s not a “bot account” to be banned) seems like overkill.
Isn’t there some simple text file or nix expression somewhere for this?
For reference, that is what their API is for, you’d create an oauth application and get an API key that way, then query using About the GraphQL API - GitHub Docs. You’d probably also want to implement some kind of cache, so you don’t need to go through all issues every time (both for your performance and to avoid rate-limiting).
Still means creating an account just for API purposes, but those aren’t forbidden on the service; they’re aware their repositories are used for, well, automation. From their ToS:
You must be a human to create an Account. Accounts registered by “bots” or other automated methods are not permitted. We do permit machine accounts:
A machine account is an Account set up by an individual human who accepts the Terms on behalf of the Account, provides a valid email address, and is responsible for its actions. A machine account is used exclusively for performing automated tasks. Multiple users may direct the actions of a machine account, but the owner of the Account is ultimately responsible for the machine’s actions. You may maintain no more than one free machine account in addition to your free User Account.
I think this may end up being relevant, because most likely someone will have to go through this process to create data in the format you’re looking for
Yeah, my experience with platform companies is that relying on this is extremely fragile. They can change the “rules of the game” for maintaining bot accounts at any time, and often are under pressure from their legal department to do so (they want evidence that a human was involved so they can argue that the continued use amounted to acceptance of the latest Terms and Conditions Update).
Expect CAPTCHAs to spring up out of nowhere, and breakage to ensue.
Always better to enter the data in some simple format, like text file git commit, and push from there to the PlatformCo API. After all, there’s currently a human using github’s website or API to open these issues; it creates no new points of failure to have that person instead enter the data into a text file and run some “push text file to github issues API” script.
I strongly encourage nixpkgs to consider having some simple text file somewhere be the upstream source of this data.
Also github’s restriction to “one bot account per human” is problematic. The process that pulls these data for me needs to run on a machine that I cannot trust with my primary github login credentials. So I would have to burn my “one bot account” allocation for this.
What happens if I’m involved in another project that needs the same thing?
So, I tried using git-bug since it has already implemented all of the github-issues-API glue. The results were very discouraging. Github’s API barfed at me several times with opaque error messages:
Something went wrong while executing your query. Please include `B92C:0B66:10A4006:11DC69B:61FD9A06` when reporting this issue.
After about a day of fighting with it, github started ratelimiting me:
rate limiting: Github GraphQL API rate limit. This process will sleep until 2022-02-05 22:31:25 +0000 UTC.
That’s a 30-minute sleep time. At this rate it’s going to take an eternity to get the bugs copied out locally.
Folks, please reconsider the decision to use github’s proprietary database format as the ultimate upstream source for security notifications. It really is not suited for this.
Matching Nix package names to NVD products is currently done via a coarse heuristic
I dunno, given my own experience as a human trying to match nix package names against my formerly-debian-adjusted brain, I am not super enthusiastic about this. I guess it’s better than nothing though.
It seems really sad that nixpkgs has a security team doing this job, but the output they’re producing is getting locked up in a proprietary database (github issues) that freaks out when you try to make a local copy of it.
