Static libraries discarded?

[charles-dyfis-net]

First section is backstory – feel free to skip.

Backstory

I’m trying to build the smallest possible system image with nwipe and its dependencies. I’ve posted a first pass on github, but that’s still much smaller than something truly minimal – its sizes are as follows:

size	package	comment
12M	/nix/store/fp89w37pzcdz7a9bj1w7yxmnn44ba5xv-ncurses-6.1	…where most of the space is
3.7M	/nix/store/v88c4w9i9xg3mca894v9psjzvb2c3q8m-musl-1.1.20
656K	/nix/store/2422zwzf1ny2lz3ck0qdjmsiy5mzqz2w-parted-3.2
76K	/nix/store/4hg3x3dvld1angf8jrmx3jnq3055xzik-nwipe-0.25	…what we’re here for

12MB for ncurses is an obvious place to focus – by making it static, we could let the compiler include only the actually useful parts in the final image.

---

Building a static ncurses

…so, I’m trying to do that like so:

{ pkgs ? import <nixpkgs> { overlays = [(self: super: {systemd = null;})];}}:

let

  staticPackage = pkg: pkg.overrideAttrs(oldAttrs: {
    dontDisableStatic = true;
    enableSharedExecutables = false;
    configureFlags = oldAttrs.configureFlags ++ [ "--without-shared" "--disable-shared" "--enable-static" ];
  });

in

(staticPackage pkgs.pkgsMusl.ncurses).overrideAttrs(oldAttrs: {
  postInstall = ''
    (set -x; mv -- lib/*.a $out/lib)
  '';
})

Output generated by that set -x shows the following command being run during build:

+++ mv -- lib/libformw.a lib/libmenuw.a lib/libncurses++w.a lib/libncursesw.a lib/libpanelw.a /nix/store/2z2wxn7k3qgzkvd3v9nfkdk6jlv0z12k-ncurses-6.1/lib

…however, the generated build doesn’t actually contain any of the lib*.a files:

$ find /nix/store/2z2wxn7k3qgzkvd3v9nfkdk6jlv0z12k-ncurses-6.1/lib/ -type f
/nix/store/2z2wxn7k3qgzkvd3v9nfkdk6jlv0z12k-ncurses-6.1/lib/terminfo

Where should I be looking to understand this behavior? And is there generally a different/better approach I should be using instead?

[Moredread]

Just out of interest, what are you planning to use it for?

[charles-dyfis-net]

Glad to answer – as aforementioned, the program I’m ultimately trying to build here is nwipe.

The intent is to build a standalone EFI executable that, when executed, wipes all drives attached to a device (for hardware with sensitive data being deprovisioned). Once I’ve got a standalone nwipe, the next step will be combining that with some of the tools currently used to build a stage-1 initrd, and thus to build a cpio archive usable as an initramfs; given that, the next step is to use objcopy to generate a single binary containing a kernel, kernel command line, and the initrd in question; after that, the only remaining piece is to deal with generating a signed version of that binary. I’m presently optimistic that this can all be described as one nix expression, once I’ve got the language down.

Since some of the devices that need to be supported have EFI boot partitions that are quite small, size matters in this context.

(Indeed, I’m probably going to also be supplementing the staticPackage function with a noBinaries function removing the /bin directory from the dependencies we’re building only for their libraries; no point to having the parted executable or the various tools that are generated along with ncurses).

[lheckemann]

(sorry, neither of my comments are actual answers)

@dtzWill’s ALLVM work might be relevant for size-minimisation/unikernel work, iirc it actually does better than both static and shared-library builds at minimising size by combining a level of sharing even greater than that achieved by shared libraries with the optimisation opportunities of static builds. As far as I understand.

Also not a real answer to your question, but have you considered deploying the standalone wipe tool via network boot? That seems like it might be more efficient:

• Permissible size increases (one would hope) from EFI partition size to RAM size, thus perhaps eliminating your need for intense size optimisation. GitHub - cleverca22/not-os: An operating system generator, based on NixOS, that, given a config, outputs a small (47 MB), read-only squashfs for a runit-based operating system, with support for iPXE and signed boot. might be relevant as a more lightweight alternative to nixos’s netboot
• No reliance on any particular contents of the hard drive or need to add a USB stick

[charles-dyfis-net]

Definitely a lot to think about there, and some of those suggestions might warrant follow-up (I’d seen early articles on ALLVM, but had it mentally catalogued as an academic experiment rather than production-ready tooling). That said, netboot doesn’t work with my deployment model, and I am still interested in an answer to the original question.

[bennofs]

This is still not a full answer as I am not 100% up to date with current nixpkgs development but you might want to look into multiple outputs, it might be that the static libraries are in another output (maybe .static) than the default one.

[matthewbauer]

Yes, it gets confusing because there are multiple ways to do static. I think the dontDisableStatic and enableSharedExecutables are a little bit hacky - we don’t know about them at eval time. Some packages have special tricks to avoid big outputs.

The culprit for ncurses is here:

NixOS/nixpkgs/blob/master/pkgs/development/libraries/ncurses/default.nix#L140

You should be able to do an override:

  staticPackage = pkg: (pkg.override { enableStatic = true; });

Eventually I want to publish an official “static” overlay that gets everything statically built. Some discussion here: Fully static Haskell executables - overview issue · Issue #43795 · NixOS/nixpkgs · GitHub. Some resistance to doing that, at least officially though.

[charles-dyfis-net]

For anyone interested in the finished product, generating a 929KB static nwipe binary —

{ pkgs ? import <nixpkgs> { overlays = [(self: super: {systemd = null;})];}}:

let

  staticOptions = pkg: pkg.overrideAttrs(oldAttrs: {
    dontDisableStatic = true;
    enableSharedExecutables = false;
    configureFlags = (oldAttrs.configureFlags or []) ++ [ "--without-shared" "--disable-shared" ];
  });
  staticPackage = pkg: (staticOptions pkg).override { enableStatic = true;};

  utillinux = (staticOptions pkgs.pkgsMusl.utillinux);
  ncurses = (staticPackage pkgs.pkgsMusl.ncurses);

  parted = ((staticPackage pkgs.pkgsMusl.parted).override {
    inherit utillinux;
    lvm2 = null;
  });

  nwipe = ((staticOptions pkgs.pkgsMusl.nwipe).override {
    inherit ncurses parted;
  }).overrideAttrs(oldAttrs: {
    NIX_CFLAGS_COMPILE = (oldAttrs.NIX_CFLAGS_COMPILE or []) ++ [ "-static" ];
    NIX_LDFLAGS = (oldAttrs.NIX_LDFLAGS or []) ++ [ "-static" ];
    buildInputs = (oldAttrs.buildInputs or []) ++ [ utillinux ];
    patches = [ ./nwipe-parted-missing-blkid-dep.patch ];
  });

in

  nwipe

…where ./nwipe-parted-missing-blkid-dep.patch is the patch also posted at Consider also explicitly depending on libblkid · Issue #57 · martijnvanbrummelen/nwipe · GitHub.

My conceptual model still needs some work here – it feels like there should be a way to only override enableStatic where it would be accepted, and I still haven’t really properly internalized the difference between attributes and arguments; regardless, this is significant improvement over where I was before, and the assistance ya’ll have offered is much appreciated.

[vcunat]

My long-term feeling is that multiple outputs aren’t really suitable for static libraries, and I’d rather see an option, i.e. use e.g. something like zlib.override { isStatic = true; }.

Of course, we would have to investigate+RFC some such style and start consistently using it…

Here’s one reason, another one is that also pkg-config and some other tools only support one library directory per package.