Radically improving Nix/NixOS security with Fil-C

Development
1

To introduce Fil-C, since it’s quite new:

Fil-C is a fanatically compatible memory-safe implementation of C and C++. Lots of software compiles and runs with Fil-C with zero or minimal changes. All memory safety errors are caught as Fil-C panics. Fil-C achieves this using a combination of concurrent garbage collection and invisible capabilities (InvisiCaps). Every possibly-unsafe C and C++ operation is checked. Fil-C has no unsafe statement and only limited FFI to unsafe code.

  • Memory Safety: Advanced runtime checks to prevent exploitable memory safety errors. Unlike other approaches to increasing the safety of C, Fil-C achieves complete memory safety with zero escape hatches.
  • C and C++ Compatibility: Your C or C++ software most likely compiles and runs in Fil-C with zero changes. Many open source programs, including CPython, OpenSSH, GNU Emacs, and Wayland work great in Fil-C. Even advanced features like threads, atomics, exceptions, signal handling, longjmp/setjmp, and shared memory (mmap style or Sys-V style) work. It’s possible to run a totally memory safe Linux userland, including GUI, with Fil-C.
  • Modern Tooling: Compiler is based on a recent version of clang (20.1.8), supports all clang extensions, most GCC extensions, and works with existing C/C++ build systems (make, autotools, cmake, meson, etc).

I managed to figure out how to build Fil-C’s from source as a Nix derivation: see my Fil-C flake. It has several working memory safe output packages: bash, coreutils, sed/grep/awk, OpenSSL, SQLite, ncurses, Nethack, tmux, Perl, Tcl, nano, and even Kitty-DOOM which runs smoothly even over SSH, after I patched John Carmack’s custom allocator from 1993 so it uses 8 byte alignments, since Fil-C crashes immediately if you even try to read a misaligned pointer.

If you cachix use filc you can immediately try it out, otherwise building the toolchain will take around an hour (YMMV).

mbrock@igloo:~$ nix run github:mbrock/filnix#sqlite
SQLite version 3.48.0 2025-01-14 11:05:00
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite>

Here compiling an awful C program in the dev shell:

mbrock@igloo:~$ nix develop github:mbrock/filnix
Fil-C 0.673 clang version 20.1.8

  clang    /nix/store/y3g4zb1bj9xbzk90qi24zal9kvk66wr1-filc-cc-wrapper-/bin/clang
  pizlo    /nix/store/kly8ckcgy2qp1l8bh827bklqkcsc4ffx-libpizlo-git
  glibc    /nix/store/h2i130kriv09kddm0mqq6w858hxxyzqx-libmojo-2.40
  libc++   /nix/store/nfwkr16gjss8nhji4p2dn2baqbmazr0i-filc-libcxx-git

(nix:filc-dev-env) mbrock@igloo:~$ clang -v
Fil-C 0.673 clang version 20.1.8
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /nix/store/41qxs3gbk4gvf2pdwg0q7k9lzfha8jch-filc0-git/bin
Build config: +assertions
Found candidate GCC installation: /nix/store/kxm6wzygmj1439ylqdgyl9zqjgf10dy7-gcc-14.3.0/lib/gcc/x86_64-unknown-linux-gnu/14.3.0
Selected GCC installation: /nix/store/kxm6wzygmj1439ylqdgyl9zqjgf10dy7-gcc-14.3.0/lib/gcc/x86_64-unknown-linux-gnu/14.3.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
[...some ugly harmless warnings I should fix...]
(nix:filc-dev-env) mbrock@igloo:~$ cat > foo.c
void main () { int foo[100]; foo[101] = 0; }
(nix:filc-dev-env) mbrock@igloo:~$ clang foo.c -o foo
[...warnings, complaints...]
2 warnings generated.
(nix:filc-dev-env) mbrock@igloo:~$ ./foo
filc safety error: cannot write pointer with ptr >= upper.
    pointer: stack_optimized(offset=404,size=400)
    expected 4 writable bytes.
semantic origin:
    <somewhere>: main
check scheduled at:
    <somewhere>: main
    ../sysdeps/nptl/libc_start_call_main.h:58:16: __libc_start_call_main
    ../csu/libc-start.c:161:3: __libc_start_main
    <runtime>: start_program
[1662962] filc panic: thwarted a futile attempt to violate memory safety.
Trace/breakpoint trap (core dumped)

If you are familiar with Nixpkgs cross compilation and toolchain bootstrapping, I hope this catches your attention! Because I really am not familiar with it, and there’s something about the recursive fixpoints and whatnot that seems to just completely fry my brain.

But I think we should aim to integrate Fil-C as a distinct ABI with a platform called x86_64-unknown-linux-filc or something, and work towards a pkgsCross.filc. Doesn’t that make sense?

Wouldn’t this be quite an extraordinary benefit?

It seems that for example NixOS could quite easily be the first distribution beyond Fil-C’s own Linux-From-Scratch distribution to offer completely memory safe versions of OpenSSH, bash, etc. Other distributions will find it more difficult to integrate two completely incompatible ABIs in the same FHS sysroot. So Nix truly shines here.

Fil-C does require patching many programs. I wanted to know how the patches were actually done, so I used a bunch of Claude subagents to simultaneously read through all of the combined patches that I extracted from the upstream ports collection, then summarized those reports into Fil-C Porting Patterns: A Technical Analysis, where you can learn about the changes needed to build e.g. Perl, Python, parts of systemd, and so on.

I came across Fil-C and when I noticed the peculiar difficulties around the ABI mismatch, and their different approaches to coping with it, it seemed like a match made in heaven for Nix, and I thought somebody has to get started on it, so I did. I’d love to hear what you all think. And if anybody wants to try it, fork it, fix it, merge it, point me to secret good resources, or even hop on a call and exchange some knowledge, that’d all be awesome.

(I also hang out on the Fil-C Discord where Filip Pizlo is also responsive and helpful if anyone wants to chat.)

22 Likes
2

By the way, speaking of complex legacy C/C++ daemons running as root, it would be very interesting to build Nix itself with Fil-C.

1 Like
3

For some usecases. But it is too slow for regular desktop usage

2 Likes
4

That doesn’t mean anything. What is “regular”? What is “desktop”? What are you actually saying?

This isn’t a constructive reply, it’s middlebrow dismissal.

4 Likes
5

Those are a lot of fantastic claims. Assuming they’re all true, where’s the catch? Surely everybody would have ditched gcc and started using Fil-C if it were just that easy.

It seems to be basically enforcing all memory safety checks at runtime, so probably the catch is that it’s very slow. The “manifesto” claims 1.2x slower on average, which feels very optimistic.

I’d like to see some benchmarks.

3 Likes
6

They’re not “fantastic” claims. Yes, the catch is that it’s slower. Benchmarks are available. Nobody is trying to deceive you. I’m not saying we need to immediately replace GCC with Fil-C. I’m saying we can use Fil-C to get memory safe versions of some programs, selectively, when that makes sense and when the benefits outweigh the costs. You can try it for yourself. That’s why I made this and why I posted it. See if Nethack is too slow. Compression with xz runs at around 1.5x, I believe. If you don’t care about memory safety, or you don’t think it’s ever worth paying a performance price to eliminate memory safety errors in critical software, this whole thing is uninteresting to you, and that’s fine!

4 Likes
7

If anyone’s interested in some notes on Fil-C from a decently respected source, Daniel J. Bernstein, check out his notes from a few days ago. He’s actively contributing to the project. He just finished a Debian bootstrapping script a couple of hours ago.

2 Likes
8

Where is the wayland/x11 stuff and benchmarks for it? You know, the “regular” “desktop” stuff.

9

I dunno, here’s a video demo.

You don’t have to run your whole GNOME desktop with Fil-C, dude. If it’s too slow, okay, I guess don’t use it, or wait for someone to optimize it.

OK? Can we talk about something more interesting than FUD?

3 Likes
10

this is super cool! this could be a nice tool to target specific pieces of software on a given system, if the user was interested

are you planning on pushing this forward? is the idea to eventually make this user friendly enough to override a stdenv or something?

great work! keep it up :rocket:

4 Likes
11

Thank you!

Yes, the flake right now has not just a plain compiler package, but a toolchain definition—I’ll just show it verbatim:

    # Fil-C toolchain with full C and C++ support
    filcc = base.wrapCCWith {
      cc       = filc-cc;
      libc     = filc-sysroot;
      libcxx   = filc-libcxx;
      bintools = filc-bintools;

      extraBuildCommands = ''
        echo "-Wno-unused-command-line-argument" >> $out/nix-support/cc-cflags
#        echo "-nostdinc++" >> $out/nix-support/libcxx-cxxflags
#        echo "-isystem ${filc-libcxx}/include/c++" >> $out/nix-support/libcxx-cxxflags
        echo "-L${filc-libcxx}/lib" >> $out/nix-support/cc-ldflags
        echo "-gz=none" >> $out/nix-support/cc-cflags
      '';
    };

This is the kind of thing that stdenv expects to see as the C/C++ toolchain, and overriding stdenv is in fact how the flake’s ports are defined:

    filenv = base.overrideCC base.stdenv filcc;
    withFilC = pkg: pkg.override { stdenv = filenv; };

But this is, like, “one layer” of replacing the stdenv, if that makes sense—it doesn’t by itself cause dependencies and transitive dependencies to use Fil-C, which is why (for example) my Nethack package looks like

  nethack = port base.nethack {
    deps = { inherit ncurses; };
  };

where port is a little helper that uses withFilC. And this is why it would be so cool to integrate with the cross compilation system—that would take care of exactly this.

2 Likes
12

They’re not “fantastic” claims.

Well, eliminating all memory errors from C code without changing it is a fantastic claim, I mean in a positive way.

Yes, the catch is that it’s slower. Benchmarks are available.

I’m not trying to dismiss you as a fraud, I was just wondering why I’ve never heard of this project before, ok?

Compression with xz runs at around 1.5x, I believe.

That’s way better than I expected.

C/C++ daemons running as root, it would be very interesting to build Nix itself with Fil-C.

Yes, but are also better and easier targets: stuff that’s doing networking and where performance is not really an issue, for example CUPS comes to mind.

4 Likes
13

Ah, yes, fantastic in a positive way, I like that. :joy:

It’s a very new project! The author has a serious resumé—he’s been the lead of WebKit’s runtime and GC, he’s published a bunch of novel research around high-performance concurrent GCs, etc. It’s not like some weird crackpot perpetual motion machine. It’s building on deep familiarity with a bunch of earlier attempts at C/C++ memory safety with different priorities; nobody’s really taken the goal of compatibility so seriously before.

This presentation from the REBASE 2024 ACM SIGPLAN conference last year is a great comprehensive introduction with a live demo that gives a pretty clear idea of what Fil-C really is.

Ah, CUPS, that’s a great idea! It’s really one of the amazing things about Fil-C that Filip is seriously interested in compatibility with the actual legacy of system software, like look at the latest commits to his repo and you’ll see he’s currently focused on getting large parts of PAM to work so you can have memory safe auth for real SSH servers.

3 Likes
14

If we are talking about the catches, I think the memory consumption impact is also pretty significant, no? So speaking of Nix, daemon probably (it needs privileges and it is also the slowest in syscalls), evaluator is probably better to sandbox.

Of course having two builds of Nix is actually what Nix is good at

1 Like
15

I think there is certainly potential for this, but my only concern is if this was adopted, how to ensure that our patches stay up to date with upstream fil-c, considering that the upstream says they have a lot of patches for programs, and how do we ensure that those patches are only applied when necessary(we don’t want to patch for fil-c specific things when not compiling for it, of course)? This is possible, but especially considering fundamental packages such as musl, glibc, openssh, bash, etc, those are packages that are not only critical(and thus should accept changes much more stringantly), but also cause large rebuild chains, and thus rebuilds of fil-c require large rebuilds of those package chains. Finally, my last issue is from a bootstrapping perspective. Nix’s bootstrapping chain for new architectures is much different from that of debian, where we cannot really rely on other architectures’ packages existing to pull from. Therefore, that’s something that for me, would be critical to fil-c being more than just “another toolchain”, for lack of a better term(and this isn’t to disparage the project, as I think it is a fundamentally good idea to have nixpkgs be flexible to try new approaches like this).

Excited to see where this goes!

4 Likes
16

Thanks! The point about patch maintenance is good and a general issue for Fil-C seen as a kind of platform and not just a compiler. I think it’s not yet clear to anyone how the patch set would be maintained. Fil-C itself probably doesn’t have the resources to maintain a proper distribution with a security team and so on. If it gains adoption probably some upstream projects (openssh, etc) will just merge Fil-C support with incompatible changes behind conditional compilation, etc. But I don’t know!

The non-yolo glibc changes are the most invasive, that’s more like a fork maintained by Fil-C and can never be applied in any other situation. Note that many packages need no changes: my flake builds unmodified versions of coreutils, ncurses, Tcl, Lua, SQLite, zlib, and so on.

I’m not familiar with how Nixpkgs bootstrap tarballs are actually defined and managed, I’ll have to look into it. My hunch is that it should be relatively straightforward to make a seed tarball derivation.

1 Like
17

I should probably mention that Fil-C generally is not stable, so this is all tentative and nowhere near mainstream default integration, and I don’t really know if it makes sense to make any kind of pull requests to Nixpkgs, although I think Nixpkgs is quite welcoming to unstable experimental stuff.

1 Like
18

I recommend reading the platform support tiers RFC.

It looks like this needs its own libc. That’s actually simpler! A package that is only needed by some exotic use case and not even built on the classical platforms is cheaper than patches to the critical parts of the stdenv.

Adding yet another cross compilation definition is a pretty low threshold. Maybe it’s a good idea to have at least two co-maintainers, but hopefully it is not intrusive, so can be accepted easily.

Once you have a properly-normal-cross-compilation set with something arguably useful in it (dunno, CUPS that should be able to take the hit in both RAM and CPU, but is often exposed to pretty open networks), you can argue you are approaching Tier 4. Then the question is what are the patches necessary and how compatilble they are. If there are good arguments that this is a cleanup of non-standard assumptions (even if the assumptions are always true on x86_64 and aarch64 and Linux and Darwin and FreeBSD), maybe Nixpkgs package maintainers will agree to apply them unconditionally (I guess it also depends on how upstream conversations go, and what is the impact on normal platforms). Otherwise you’ll need to fork the expressions.

1 Like
19

This feels optimistic, we already barely-maintain pkgsMusl and co, adding even more maintenance load that isn’t going to get tested in CI, is harder to maintain than what we already (don’t) maintain, and that not-many people have the knowledge to maintain sounds like… a recipe for supbar results.

20

Presumably the interested people are different, so I’d expect no real splitting-effort effects and the outcome a bit below pkgsMusl, which is still enough to have a few particularly well-fitting packages usable from there.

2 Likes