Hi everyone,
for this release, I hacked together https://zh.fail and since there is no GitHub issue tracker to give feedback, I created this post.
Whatever your feedback is, feel free to post it here. General (dis)pleasure, questions about how the numbers are generated, page improvements, feature wishlist, …. Just everything.
Thank you for early testing by the people I asked, mainly @hexa (by far the most feedback 
), @ajs124 and @Mic92.
Technical details about the page
I hope this explains how I do things and this should make it very clear what’s easy to implement and what is not.
The page is built on our company GitLab in a pipeline running on our CI server. The pipeline scrapes data and formats it into a static page that is then served via GitLab pages. The page being statically generated allows me to format the data pretty much any way I want because each HTTP request from a client is almost free.
That doesn’t mean there is no CPU time going into this. Each job takes from a couple of minutes (no new eval) to way over 2h. The time is spent with crawling Hydra and with interacting with git and nix-instantiate to find package maintainers.
The initial idea was to generate the data and sed it into a static HTML page which is why I started with bash and a large sed call. This unfortunately escalated so I now have an overly large bash script that calls out into Python scripts a couple of thousand times to parse the HTML coming from Hydra (the API doesn’t have all data I need). Everything is highly cached (as far as possible) to minimize the damage that I do from scraping Hydra. I did also inform the Infra team (namely @grahamc) to let them know what I’m doing and to give them an easy way to contact me when it gets too excessive.
The process is: Ask Hydra about evals of the two jobsets, filter out unfinished evals, crawl the entire evaluation (all builds and their state). Then I sum up all non-successful builds and check out the respective commits in a local nixpkgs checkout. I evaluate meta.maintainers of each failed package and use this to build the per-maintainer pages (If you look for a maintainer called _, you will get all failed packages that don’t have a maintainer).
I do not plan to make the source code public available as of writing. One reason is that the code is really horrible and I’m ashamed of it and the other reason is that I want to keep other people from scraping Hydra as well. This might change but is not planned right now and will have to be coordinated with the Infra team first. If you want parts of the code, feel free to ask about specific details.
How the data is generated
Some of the data is a bit hard to define so I just did it in a way that seemed logical to me. If you think I got something wrong - this is the post to complain If I use examples here, I use the trunk-combined jobset, I also scrape the master jobset for Darwin.
- The evals are found by parsing the HTML of Hydra. I parse the pills on the right to find evals that are fully finished. This is also where the counts on the graph come from.
- I crawl the entire evaluation (all tables from the eval view) to find the status of all builds (hover on the icon), the build ids and the package names.
- I calculate the per-platform fails from the data gathered in the previous step, basically summing up the non-successful builds per
system. - I check out the revision that Hydra tried to build in a local nixpkgs checkout and trace the attributes to a list of maintainers with a local
nix evalcall. - I use this data to build the per-maintainer views. There is actually a Hydra bug that might sort indirect failures as direct failures, this is known.
- I look for the most important dependencies. The code is very roughly based on what @samueldr did with the nix-review-tools but I do it in Python and BS4 rather than Ruby.
- I sed everything into index.html and copy it to the
public/directory.
)