How to override openssl (Q&A)

Help
1

Context

Here is the shell.nix for a project

let
    # get whatever global nixpkg version the user has installed
    nixpkgs = import <nixpkgs> {};    
in
    # create a shell
    nixpkgs.mkShell {
        buildInputs = [
            # some basic unix networking commands like `ping`
            nixpkgs.unixtools.arp
            nixpkgs.unixtools.ifconfig
            nixpkgs.unixtools.netstat
            nixpkgs.unixtools.ping
            nixpkgs.unixtools.route
        ];
    }

Problem

On MacOS, all of those depend on the vulnerable openssl_1_0_2

error: Package ā€˜openssl-1.0.2uā€™ in /nix/store/3a5y4fwmnvpfgykvr79qis1nd7f7x3w0-source/pkgs/development/libraries/openssl/default.nix:140 is marked as insecure, refusing to evaluate.


Known issues:
 - Support for OpenSSL 1.0.2 ended with 2019.

You can install it anyway by whitelisting this package

[ Iā€™ve been whitelisting the package, but would like to do better ]

Additional Info

I found the unixtools package (source here)
(I found it by cloning the nixpkg github repo and searching it for unixtools)

The unixtools package is using pkgs.darwin.network_cmds
Here is a relevant code snippet of what I mean:

    ifconfig = {
      linux = pkgs.nettools;
      darwin = pkgs.darwin.network_cmds;
    };

From there I searched for the darwin.network_cmds.
This is where the actual openssl dependency occurs:

What has been attempted

Sources that have been read:

My first attempt, following the nix pill

let
    nixpkgs = import <nixpkgs> {};
    # similar to pkgs.graphviz = pkgs.graphviz.override { xlibs = null; };
    fixedOpenssl = nixpkgs.unixtools.ifconfig.override {
        openssl_1_0_2 = nixpkgs.openssl_1_1; 
    };
in
    nixpkgs.mkShell {
        buildInputs = [
            fixedOpenssl
        ];
    }

Other attempts included openssl =, trying to use overlays, and trying to simply globally install openssl_1_1.

2

The Override (Answer pt.1)

For anyone else in a similar situation, here is how to do the override.
(although the override isnā€™t the end of the story)

The original source (pkg.darwin.network_cmds) itself must be overridden, not the bundled package.

let
    nixpkgs = import <nixpkgs> {};
    # similar to pkgs.graphviz = pkgs.graphviz.override { xlibs = null; };
    fixedOpenssl = nixpkgs.darwin.network_cmds.override {
        openssl_1_0_2 = nixpkgs.openssl_1_1; 
    };
in
    nixpkgs.mkShell {
        buildInputs = [
            fixedOpenssl
        ];
    }
3

That override (not too unexpectedly) results in an error.

Real Question

What kinds of general approaches do you (anyone in the nix community) have to solving these kinds of errors? Error log posted below.

This is on MacOS 10.12.6 so mentions of 10.11.6 could be a potential cause.
(I have a strong background in C++ so feel free to talk about struct/pointer errors if thats relevant)

/nix/store/00zgg42a8l2jplbrk6fxxdyakan4ggg2-Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c -arch x86_64 -isysroot /nix/store/qjk65a8dx8hq9rm5cjdz4d4gsvl3g5bh-SDKs/MacOSX10.12.sdk -mmacosx-version-min=10.12 -fmessage-length=0 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit=0 -std=gnu99 -fmodules -Wnon-modular-include-in-framework-module -Werror=non-modular-include-in-framework-module -fmodules-prune-after=345600 -fmodules-prune-interval=86400 -fmodules-cache-path=./ModuleCache -DUSE_RFC2292BIS=1 -D__APPLE_USE_RFC_3542=1 -D__APPLE_API_OBSOLETE=1 -fasm-blocks -fstrict-aliasing -fpascal-strings -Os -g -Wno-trigraphs -Wno-empty-body -Winfinite-recursion -Wno-deprecated-declarations -Wno-missing-field-initializers -Wno-newline-eof -Wno-missing-prototypes -Wpointer-sign -Wno-return-type -Wswitch -Wno-four-char-constants -Wno-conversion -Wno-bool-conversion -Wno-constant-conversion -Wno-enum-conversion -Wno-int-conversion -Wno-sign-conversion -Wno-shorten-64-to-32 -Wno-missing-braces -Wparentheses -Wno-shadow -Wno-uninitialized -Wno-unknown-pragmas -Wunused-function -Wno-unused-label -Wno-unused-parameter -Wunused-value -Wno-unused-variable -I/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/Intermediates/network_cmds.build/Release/unbound.build/unbound.hmap -I/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/Products/Release/include -I/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/Intermediates/network_cmds.build/Release/unbound.build/DerivedSources/x86_64 -I/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/Intermediates/network_cmds.build/Release/unbound.build/DerivedSources -F/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/Products/Release -Wall -MMD -MT dependencies -MF /private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/Intermediates/network_cmds.build/Release/unbound.build/Objects-normal/x86_64/keyraw.d -c /private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c -o /private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/Intermediates/network_cmds.build/Release/unbound.build/Objects-normal/x86_64/keyraw.o
/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:220:5: error: incomplete definition of type 'struct dsa_st'
        dsa->p = P;
        ~~~^
In file included from /private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:14:
In file included from ./unbound/ldns/keyraw.h:27:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ssl.h:17:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/comp.h:16:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/crypto.h:25:
/nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ossl_typ.h:107:16: note: forward declaration of 'struct dsa_st'
typedef struct dsa_st DSA;
               ^
/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:221:5: error: incomplete definition of type 'struct dsa_st'
        dsa->q = Q;
        ~~~^
In file included from /private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:14:
In file included from ./unbound/ldns/keyraw.h:27:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ssl.h:17:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/comp.h:16:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/crypto.h:25:
/nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ossl_typ.h:107:16: note: forward declaration of 'struct dsa_st'
typedef struct dsa_st DSA;
               ^
/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:222:5: error: incomplete definition of type 'struct dsa_st'
        dsa->g = G;
        ~~~^
In file included from /private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:14:
In file included from ./unbound/ldns/keyraw.h:27:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ssl.h:17:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/comp.h:16:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/crypto.h:25:
/nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ossl_typ.h:107:16: note: forward declaration of 'struct dsa_st'
typedef struct dsa_st DSA;
               ^
/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:223:5: error: incomplete definition of type 'struct dsa_st'
        dsa->pub_key = Y;
        ~~~^
In file included from /private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:14:
In file included from ./unbound/ldns/keyraw.h:27:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ssl.h:17:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/comp.h:16:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/crypto.h:25:
/nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ossl_typ.h:107:16: note: forward declaration of 'struct dsa_st'
typedef struct dsa_st DSA;
               ^
/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:278:5: error: incomplete definition of type 'struct rsa_st'
        rsa->n = modulus;
        ~~~^
In file included from /private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:14:
In file included from ./unbound/ldns/keyraw.h:27:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ssl.h:17:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/comp.h:16:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/crypto.h:25:
/nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ossl_typ.h:110:16: note: forward declaration of 'struct rsa_st'
typedef struct rsa_st RSA;
               ^
/private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:279:5: error: incomplete definition of type 'struct rsa_st'
        rsa->e = exponent;
        ~~~^
In file included from /private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/unbound/ldns/keyraw.c:14:
In file included from ./unbound/ldns/keyraw.h:27:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ssl.h:17:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/comp.h:16:
In file included from /nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/crypto.h:25:
/nix/store/qh0x9pmjy9m5ghs8naznmnhv4czc3cak-openssl-1.1.1g-dev/include/openssl/ossl_typ.h:110:16: note: forward declaration of 'struct rsa_st'
typedef struct rsa_st RSA;
               ^
6 errors generated.

** BUILD FAILED **

The following build commands failed:
    CompileC /private/tmp/nix-build-network_cmds-osx-10.11.6.drv-0/network_cmds-481.20.1/Intermediates/network_cmds.build/Release/unbound.build/Objects-normal/x86_64/keyraw.o unbound/ldns/keyraw.c normal x86_64 c com.apple.compilers.llvm.clang.1_0.compiler
(1 failure)
builder for '/nix/store/i4xvfii01mj3vg97lxvqg24c8c3qwhnw-network_cmds-osx-10.11.6.drv' failed with exit code 1
error: build of '/nix/store/i4xvfii01mj3vg97lxvqg24c8c3qwhnw-network_cmds-osx-10.11.6.drv' failed
4

My understanding is that openssl 1.1 is not backwards-compatible with openssl 1.0.2. Looking at OpenSSL 1.1.0 Changes it explicitly mentions that openssl 1.1.0 made a lot of structs opaque, which seems the likely cause for all those ā€œincomplete definitionā€ errors. Instead clients are supposed to use accessor functions.

So basically, the only way to upgrade openssl with these tools is to update the tools themselves. These are coming from Appleā€™s open source releases, and since Apple doesnā€™t ship OpenSSL 1.1.0 or later this means theyā€™re not going to do the work to make the commands compatible with it. So weā€™d have to maintain our own patches.

1 Like