Reproducible Builds summit report

Events
#1

@Profpatsch , @lewo, @edolstra and me went to the summit this week. It was a lot of fun. We met with people from all sorts of distros; Debian, OpenSuse, Arch Linux, Alpine, and obviously Guix. Members of IPFS, QubeOS, Ocaml and Microsoft(?) were also present. We were ~50+ people total.

Some highlights from my side:

Everyone intuitively knows why reproducible builds are important but it’s actually quite hard to explain why. I still don’t have an answer that is easy to explain.

The Guix team is working on bootstrapping GCC from a few bytes of binary, to a minimal scheme interpreter, TCC, an old version of GCC → until the latest GCC. And happy to share their results with other distros. @edolstra was taking a look at it.

Did you know that nix-build has a --check flat? It builds the derivations twice and tells you if the output differs. And Hydra also supports that feature but it’s undocumented.

https://reproducible-builds.org/ has a bunch of tools that could be useful for us. I think @Profpatsch packaged most of them.

Ruby gems that contain C extensions are not bit-reproducible as they contain logs. @lewo was looking into this.

The only thing I implemented in the end had nothing to do with reproducible builds. Inspired by Guix, Nix will have nicer hash mismatch messages in the next release: libstore: improve hash mismatch error messages · NixOS/nix@5e6fa90 · GitHub

Please add anything I forgot or didn’t pay attention to :slight_smile:

16 Likes
#2

The event was organized without a fixed schedule, rather the
organizers implemented a scheme wherein participants would form around
topics that interested them.

Most sessions had somebody take notes, the reports are going to be
published in a “Reports” section on the event page soon:
https://reproducible-builds.org/events/paris2018/

Here I wrote down some interesting one-on-one conversations I had over
the three days.

2018-12-11

2018-12-12

  • Hacksession: Package Matrix for Tools — reproducible-builds.org
    I hosted two hacking sessions to show & improve the distribution of
    Rep. Builds tools on various distros. Writeup here:
    Riseup Pad
    (and later in the agenda)
  • Typesystem for JSON
    Met a person who is writing a typesystem for JSON files for some
    IPFS-related project. Since I had already done something similar
    for nix expressions
    (nixpkgs/types-simple.nix at 43a5946dd3f2059f960f517f929b174f7099094d · Profpatsch/nixpkgs · GitHub)
    I explained my implementation. An interesting question that came
    up: What if you don’t want the type specifications to be literal
    functions in an untyped setting, but make them data (e.g. by
    representing them in JSON) that can only be combined in a certain
    way.

2018-12-13

  • Missed the first part, joined after lunch (Appartment search)
    It was mostly a hacking session in the afternoon.
    Since I had to organize additional appartment-related things, I
    wasn’t able to continue working on the tools matrix very much.
    The event ended with a thank-you session & continued hacking while
    people were slowly leaving.
  • Tansy
    Tansy is working on a build system similar to bazel, pants. It was
    created by ex-Googlers and used primarily for its excellent Scala
    support (because sbt is a horror show).
    I brought up the notion that build tools tend to be able to
    integrate support for different languages, but don’t play fair when
    it comes to integrate themselves into other build systems. For
    example nix assumes access to /nix, bazel uses a daemon with long
    startup time and makes it hard to inject e.g. a prebuilt protobuf.
3 Likes
#3

This is a really nice event where it’s easy to meet and discuss with a
lot of people, thanks to the non classical format as explained by
@Profpatsch.

The option mentioned by @Zimbatm is actually build-repeat 1 which
is useful to check the reproducibility of builds. But there are still
some reproducibilitly issues that could be hard to hit with this
option such as in [1,2]: the contents of the file depends on the
file order on the FS. For this kind of issues, I started to enable
disorderfs [3] in the sandbox when the build-repeat option is
set, but this is not finished yet.

Since the IPFS community was present, we also discussed about IPFS
and binary caches! Guix people seem to be pretty interested in improving
performances of their binary cache (which doesn’t rely on AWS/CDNs).
To be continued…

[1] Sort list of RECORD entries by bmwiedemann · Pull Request #5525 · pypa/pip · GitHub
[2] Sort files and test_files in specifications by cbaines · Pull Request #2524 · rubygems/rubygems · GitHub
[3] Reproducible Builds / disorderfs · GitLab

2 Likes
#4

This is a great change; thank you @zimbatm!

1 Like
Hosted by Flying Circus.