Using duckdb shared library in elixir project?

Help
1

Hey,

I’m trying to use DuckDB inside elixir and the recommended way by Jose seems to be through adbc.

I followed this blog post about elixir/iceberg/duckdb and found that the the typical way to install it is to run:

# Change URL for your architecture: https://github.com/duckdb/duckdb/releases/tag/v1.4.0
Adbc.download_driver!(:duckdb, version: "1.4.0", url: "https://github.com/duckdb/duckdb/releases/download/v1.4.0/libduckdb-linux-arm64.zip", force: true)

This is of course problematic to do in the production with read only nix store.

I would mainly need the shared library libduckdb.so from the zip file:

$ curl https://github.com/duckdb/duckdb/releases/download/v1.4.2/libduckdb-linux-arm64.zip -O
$ unzip -l libduckdb-linux-amd64.zip
Archive:  libduckdb-linux-amd64.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
 64805880  11-11-2025 13:38   libduckdb.so
 70935094  11-11-2025 13:38   libduckdb_static.a
  1875021  11-11-2025 13:39   duckdb.hpp
   190571  11-11-2025 13:13   duckdb.h
---------                     -------
137806566                     4 files

I’m now wondering if this is already built in the duckdb nix package or some other duckdb lib?

Is there a way for me to search with filenames from nix packages?

What would be the best course of action to include this in my project?

Claude Code currently helped me to do this and it does seem way too complex for me:

{
  mixRelease,
  fetchMixDeps,
  elixir,
  writeScript,
  pkgs,
  lib,
  beamPackages,
  stdenv,
}:
let
  # DuckDB version - defined once and used everywhere
  duckdbVersion = "1.4.2";

  # Fetch DuckDB library with ADBC support built-in
  duckdbLib = pkgs.fetchzip {
    url = "https://github.com/duckdb/duckdb/releases/download/v${duckdbVersion}/libduckdb-linux-amd64.zip";
    sha256 = "sha256-W1YOf4zBQkkLxEGWyWg2dXJhWSzRrm+MWau9bvvMSQA=";
    stripRoot = false;
  };
in
mixRelease rec {
...
  postInstall = ''
        ...
        # Create drivers directory for pre-installed DuckDB library
        DRIVERS_DIR="$ADBC_LIB_DIR/priv/drivers"
        mkdir -p "$DRIVERS_DIR"

        # Copy libduckdb.so from fetched DuckDB release
        echo "Installing DuckDB library v${duckdbVersion} from ${duckdbLib}"
        cp "${duckdbLib}/libduckdb.so" "$DRIVERS_DIR/libduckdb.so"
        chmod +w "$DRIVERS_DIR/libduckdb.so"

        # Patch RPATH to include Nix store paths for dependencies
        # DuckDB needs libstdc++, libgcc_s, and libz
        ${pkgs.patchelf}/bin/patchelf --set-rpath "${pkgs.stdenv.cc.cc.lib}/lib:${pkgs.zlib}/lib" "$DRIVERS_DIR/libduckdb.so"

        echo "Installed and patched DuckDB library at $DRIVERS_DIR/libduckdb.so"
        echo "DuckDB library RPATH: $(${pkgs.patchelf}/bin/patchelf --print-rpath "$DRIVERS_DIR/libduckdb.so")"

'';

...
2

You can use nix-index to find files.

❯ nix-locate duckdb.so
nexusmods-app.out                            64,719,144 x /nix/store/swkb209ypj746an3pym3lykmk8l53rs2-nexusmods-app-0.20.2/lib/nexusmods-app/libduckdb.so
duckdb.lib                                   72,154,048 x /nix/store/pizli27fpml9bqpcfds6dvbniavhp718-duckdb-1.4.1-lib/lib/libduckdb.so

which you can xcheck with the repl

❯ nix repl -f '<nixpkgs>'
Nix 2.28.5
Type :? for help.
Loading installable ''...
Added 24872 variables.
nix-repl> :b duckdb.lib

This derivation produced the following outputs:
  dev -> /nix/store/f8w2y7py6767qgsqwcrzlj52dm358xc8-duckdb-1.2.2-dev
  lib -> /nix/store/0arq7q42rsk3f4nzrk5kdrxyiydry90r-duckdb-1.2.2-lib
  out -> /nix/store/qllwzkz5pvvnkkgcqlpxjbswn3xni4d5-duckdb-1.2.2



~                                                                                       20s ben@lamorna 09:57:00
❯ lt /nix/store/0arq7q42rsk3f4nzrk5kdrxyiydry90r-duckdb-1.2.2-lib
/nix/store/0arq7q42rsk3f4nzrk5kdrxyiydry90r-duckdb-1.2.2-lib
└── lib
    ├── libautocomplete_extension.a
    ├── libcore_functions_extension.a
    ├── libduckdb.so
...

so duckdb.lib has the .so. As to the rest I don’t know exlir :slight_smile:

2 Likes
3

That worked perfectly!

I was able to copy from there in the postInstall phase:

cp "${pkgs.duckdb.lib}/lib/libduckdb.so" "$DRIVERS_DIR/libduckdb.so"

And now the rpath patching was not needed anymore :slight_smile:

The only issue is that the available version is not the latest duckdb (1.4.2) and because of that few community extensions I created for DuckDB are not available in 1.4.1 which is the newest duckdb in nixpkgs.

I will just have to wait until it gets released :+1:

4

This is nix, why would you wait for anything? :smiley:

nix-repl> :b duckdb.overrideAttrs(old: { version = "1.4.2"; src = fetchFromGitHub rec { version = "1.4.2"; inherit (old.src) owner repo; tag = "v${version}"; hash = "sha256-2ogWmxdM9hC7qX7pTEe0oOKGYRdgxk/6z3rsRBXc70E="; }; })

This derivation produced the following outputs:
  dev -> /nix/store/2ypampxx9hhfbaa2shv7fx85m9820xvq-duckdb-1.4.2-dev
  lib -> /nix/store/1z9ld7whn25907r2y8b0hb0q22xakss0-duckdb-1.4.2-lib
  out -> /nix/store/7yxck2mg8m12k86v63fb0fqnx8fm302n-duckdb-1.4.2

Admittedly this doesn’t add the correct revision to the derivation so there might be some metadata somewhere that is wrong, but you can also fix that if it causes you an issue.

1 Like
5

Yeah that’s a good idea and I will try it it out :+1:

Would you also know the best way to cache runtime dependencies with nix?

I downloaded these in my service startup script earlier but because cloudflare was down it wasn’t possible to download them yesterday and the service initiation script failed.

This is how it works:

# Trying to use the extensions fails when it's not yet installed
$ duckdb -c "LOAD nanoarrow;"
IO Error:
Extension "~/.duckdb/extensions/v1.4.1/linux_amd64/nanoarrow.duckdb_extension" not found.

Candidate extensions: "azure", "autocomplete", "parquet", "spatial"

# Install the extension with duckdb
$ duckdb -c "INSTALL nanoarrow FROM community"

# Or download the extension with curl
$ curl https://community-extensions.duckdb.org/v1.4.1/linux_amd64/nanoarrow.duckdb_extension.gz | \
  gunzip - > .duckdb/extensions/v1.4.1/linux_amd64/nanoarrow.duckdb_extension

# Now it works
$ duckdb -c "LOAD nanoarrow;"

What would be the nix way to download multiple extensions already when building the derivation?

6

I guess it depends who/what you are targetting. If the extensions need to end up in your home directory and you use hm (or similar) then home.file.".duckdb/...".source = pkgs.somefetcher is sufficient. I just had a quick read of the duckdb website, and seems really like they are supposed to be fetched on demand, there is no provision for providing search paths or anything of that ilk, so you’d be relying on the structure of their extensions folder not changing if you chose to put it under nix control.

Maybe someone more well versed with duckdb will chime in.

7

Yeah but this is fine since the structure doesn’t change randomly in runtime but only when the versions are changing and I can even run a check in the build script (without internet access) to check that it still loads the extension from the local path or fails because it can’t connect to duckdb extensions :+1:

1 Like