Nix flamegraph or profiling tool?

Development
1

I’m having a big closure that takes too much to evaluate. And it has no IFDs. That’s weird.

I’m used to using py-spy to generate flamegraph data that I can feed into https://www.speedscope.app/ to inspect and solve Python apps performance bottlenecks.

Is there anything similar for Nix lang?

5 Likes
2

I have this zsh function

function nixFunctionCalls {
    local WORKDIR=$(mktemp -d /tmp/nix-fun-calls-XXXXX)
    nix-instantiate --trace-function-calls "$1" -A "$2" 2> $WORKDIR/nix-function-calls.trace 1>/dev/null
    ~/nixgits/nix/contrib/stack-collapse.py $WORKDIR/nix-function-calls.trace > $WORKDIR/nix-function-calls.folded
    nix-shell -p flamegraph --run "flamegraph.pl $WORKDIR/nix-function-calls.folded > $WORKDIR/nix-function-calls.svg"
    echo "$WORKDIR/nix-function-calls.svg"
}

I use it like so nixFunctionCalls . bash | xargs firefox

here’s the .folded file in speedscope

image
image2039×1305 221 KB

15 Likes
3

Is there a flakes alternative to nix-instantiate?

4

I’m trying your solution and a problem I find is that my .folded file occupies about 59 GB, so I cannot upload it to speedscope, and trying to get a local flamegraph out of it blows my machine.

Is there any other more efficient way to get a flamegraph?

5

Mine for the nixpkgs bash is only a few mb

I dont know of other ways

6

From this reply
https://github.com/NixOS/nix/issues/7179#issuecomment-1678230028

nix eval --trace-function-calls --raw ".#bash" 2>&1 | cat

For some reason it has to be piped, weird bug.

7

py-spy has the notion of samples per second. I think that’s how they produce a reasonably-sized dump.

It seems to me that nix just traces every single call, which can amount to many millions when using modules (I have them while using flake-parts and dream2nix in the same project).

It would be nice to be able to have a “samples per second” option for --trace-function-calls.

8

NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 could also be helpful

1 Like
9

Both options seem to dump useful information, but it’s not very human-readable.

Is there any tool that generates some sort of graphic out of that data?

10 
00000000: 0d1b 5b4b 0d1b 5b4b 6675 6e63 7469 6f6e  ..[K..[Kfunction
00000010: 2d74 7261 6365 2065 6e74 6572 6564 20c2  -trace entered .

Nix is putting control chars in stderr for some reason. After cleaning up the control characters and passing it through this very jank AI-generated script, flamegraph.pl is able to generate svgs.

EDIT: Nevermind, the script doesn’t produce correct results. But it should be easy enough to do.

11

Adapted to home-manager configuration, made it use nix run to support flakes and inferno for magnitudes faster svg generation:

 home.packages =
      let
        # https://discourse.nixos.org/t/nix-flamegraph-or-profiling-tool/33333
        stackCollapse = pkgs.writeTextFile {
          name = "stack-collapse.py";
          destination = "/bin/stack-collapse.py";
          text = builtins.readFile (builtins.fetchurl
            {
              url = "https://raw.githubusercontent.com/NixOS/nix/master/contrib/stack-collapse.py";
              sha256 = "sha256:0mi9cf3nx7xjxcrvll1hlkhmxiikjn0w95akvwxs50q270pafbjw";
            });
          executable = true;
        };
        nixFunctionCalls = pkgs.writeShellApplication {
          name = "nixFunctionCalls";
          runtimeInputs = [ stackCollapse pkgs.inferno ];
          text = builtins.readFile ./nix-function-calls.sh;
          checkPhase = "";
        };
      in
      [ nixFunctionCalls ];

#!/usr/bin/env zsh

WORKDIR=$(mktemp -d)

nix eval -vvvvvvvvvvvvvvvvvvvv --raw --option trace-function-calls true $1 1>/dev/null 2> $WORKDIR/nix-function-calls.trace
stack-collapse.py $WORKDIR/nix-function-calls.trace > $WORKDIR/nix-function-calls.folded
inferno-flamegraph $WORKDIR/nix-function-calls.folded > $WORKDIR/nix-function-calls.svg
echo "$WORKDIR/nix-function-calls.svg"
4 Likes
12

thanks a lot for posting it here, can you please explain to me why the pipe xargs command?

like in your example:

nixFunctionCalls . bash | xargs firefox

thanks!

13

If you pipe this to xargs firefox it will be the equivalent of firefox ./$WORKDIR/nix-functiona-calls.svg.

1 Like
14

Some additional ongoing efforts/experiment for a Nix expr evaluation profiler using tracy:

1 Like
15

in case any of you are interested, I went ahead and cobbled together the scripts @tailrecursive posted into an easily accessible nix flake. You can find it here: GitHub - crabdancing/nix-flamegraph: Various primitive scripts cobbled together to produce an easy flamegraph of nix eval

1 Like