Any tutorials/guides on Solana development?

rollschild · April 14, 2022, 1:53am

oh and in his PR that I linked above, there is a screenshot showing that he successfully ran cargo build-bpf, which I’m not sure how. Because it would give me the “read-only file system” error.
131382122-39d8c60b-f4ec-4b25-876c-8b9da9a99234

itsfarseen · April 14, 2022, 6:54am

Because it would give me the “read-only file system” error.

Their PR patches the cargo-build-bpf binary to not download the sdk to /nix/store/<some-folder>. That’s how they are able to do that.

I’m not really sure how to run this patchelf , since I would need to run this command on executables, not directories, am I right?

I’m so sorry I messed up this bit. It’s not patchelf, it’s autopatchelf from here: GitHub - svanderburg/nix-patchtools: Autopatching binary packages to make them work with Nix

I forgot which one I used but I remembered when you mentioned directories.

I think it’s enough to run autopatchelf on ./sdk, since the toolchain is symlinked to there.

rollschild · April 15, 2022, 5:25am

Okay, I’ve sorta made some progress?

I slightly modified my shellHook to set the $BPF_SDK_PATH:

          shellHook = ''
            mkdir -p ~/projects/example-helloworld/src/program-rust/sdk
            export BPF_SDK_PATH="/home/rollschild/projects/example-helloworld/src/program-rust/sdk"
            export LIBCLANG_PATH="${llvmPackages_7.libclang.lib}/lib"

            rm -rf ~/.cache/solana*
            mkdir -p ~/.cache/solana/${solana-bpf-tools-pkg.version}
            ln -s ~/projects/example-helloworld/src/program-rust/sdk/dependencies/bpf-tools ~/.cache/solana/${solana-bpf-tools-pkg.version}/bpf-tools
          '';

so that cargo build-bpf no longer fails on the “readonly file system” error.

But I’m still seeing bpf no subcommand errors.

So I git cloned the autopatchelf repo and ran nix-build to build autopatchelf into the nix-patchtools/result/bin/autopatchelf. Then I in my current project I ran this command but got the No package found that provides library: librustc_driver-a8aaff3fbd0bfff2.so error:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ../../../nix-patchtools/result/bin/autopatchelf sdk/dependencies/bpf-tools/rust/
Patching all ELF binaries in directory: sdk/dependencies/bpf-tools/rust/
Auto patching ELF binary: sdk/dependencies/bpf-tools/rust/bin/rustc
No package found that provides library: librustc_driver-a8aaff3fbd0bfff2.so

which seems weird, because sdk/dependencies/bpf-tools is just a symlink to ~/.cache/solana/v1.23/bpf-tools, which looks correct…

itsfarseen · April 16, 2022, 9:23am

I’ll give you the cause of each error messages:

bpf no subcommand:
cargo doesn’t have a +bpf subcommand. +<toolchain> is a rustup subcommand.
Non-rustup cargo is overriding the rustup cargo.
command not found: The binary that is being called has an invalid loader path. Fix it using autopatchelf.
read only filesystem: Solana sdk is trying to download the bpf sdk into /nix/store/.
Fix it by giving an explicit --bpf-sdk path.

I had messed up a bunch of things like forgetting about autopatchelf when I gave you the initial instructions. But now that I have given you everything, you should be able to use the flake.nix I shared in the first post and get it working.

itsfarseen · April 16, 2022, 9:34am

itsfarseen:

{
  description = "";
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs";
    rust-overlay.url = "github:oxalica/rust-overlay";
    flake-utils.url  = "github:numtide/flake-utils";
  };
  outputs = { self, nixpkgs, rust-overlay, flake-utils, ... }: flake-utils.lib.eachDefaultSystem (system:
    let
      overlays = [ (import rust-overlay) ];
      pkgs = import nixpkgs {
        inherit system overlays;
      };
      solana-cli = {stdenv, fetchurl, fetchzip, lib, autoPatchelfHook, pkgs }: stdenv.mkDerivation rec {
        name = "solana-${version}";
        version = "1.10.3";
        filename = "solana-release-x86_64-unknown-linux-gnu.tar.bz2";
        src = fetchzip {
          url = "https://github.com/solana-labs/solana/releases/download/v${version}/${filename}";
          sha256 = "sha256-jOUCyK7spTeEZz+h2hxfGpDkZ+2pYbyUFzlQ20c1zv4=";
        };
        nativeBuildInputs = [ autoPatchelfHook pkgs.makeWrapper ];
        buildInputs = with pkgs; [
          sgx-sdk
          ocl-icd
          eudev
          rustup
          stdenv.cc.cc
        ];

        installPhase = ''
          mkdir -p $out;
          cp -r bin $out;
          mkdir -p $out/bin/sdk/;
          ln -s "${solana-bpf-tools-pkg}" $out/bin/sdk/bpf;
          chmod 0755 -R $out;
        '';

        meta = with lib; {
          homepage = "https://docs.solana.com/cli/install-solana-cli-tools#download-prebuilt-binaries";
          platforms = platforms.linux;
        };
      };

      solana-bpf-tools = {stdenv, fetchurl, fetchzip, lib, autoPatchelfHook, pkgs }: stdenv.mkDerivation rec {
        name = "solana-bpf-tools-${version}";
        version = "1.23";
        src = fetchzip {
          url = "https://github.com/solana-labs/bpf-tools/releases/download/v${version}/solana-bpf-tools-linux.tar.bz2";
          sha256 = "sha256-4aWBOAOcGviwJ7znGaHbB1ngNzdXqlfDX8gbZtdV1aA=";
          stripRoot = false;
        };

        nativeBuildInputs = [ autoPatchelfHook ];
        buildInputs = with pkgs; [
          zlib
          stdenv.cc.cc
          openssl
        ];

        installPhase = ''
          mkdir -p $out;
          cp -r $src/llvm $out;
          cp -r $src/rust $out;
          chmod 0755 -R $out;
        '';

        meta = with lib; {
          homepage = "https://github.com/solana-labs/bpf-tools/releases";
          platforms = platforms.linux;
        };
      };
      solana-cli-pkg = (pkgs.callPackage solana-cli {});
      solana-bpf-tools-pkg = (pkgs.callPackage solana-bpf-tools {});
    in with pkgs; {
      devShell = mkShell {
        buildInputs = [
          rust-analyzer
          solana-cli-pkg 
          solana-bpf-tools-pkg
          cargo-edit
          rustup
        ];
      };
    }
  );
}

To recap:

Use the above flake.nix. It will make rustup and cargo build-bpf available.
Install a normal rust toolchain using rustup. Skip this if you already have installed. Run rustup toolchain list to see.
Remove bpf toolchain if it is shown in the list, to start from a blank slate. rustup toolchain uninstall bpf.
Run cargo build-bpf --bpf-sdk ./sdk. It will download bpf sdk to ./sdk/dependencies.
After downloading dependencies, it will try to run it, but it will fail, because we haven’t run patchelf on it. This is normal.
Run autopatchelf ~/.cache/solana. The stuff that got downloaded in ./sdk is actually downloaded to ~/.cache/solana and symlinked from there.
Symlink the following two things to ./sdk
- ln -s /nix/store/c11gzzi5ax9b8kmqrizrazjil4m8c4dl-solana-1.10.3/bin/sdk/bpf/env.sh ./sdk/
- ln -s /nix/store/c11gzzi5ax9b8kmqrizrazjil4m8c4dl-solana-1.10.3/bin/sdk/bpf/scripts ./sdk/
  The hash might be different in your system. Run fd or find or ls | grep in /nix/store to find the solana folder and substitute it’s path in place of c11gzz...c4dl-solana-1.10.3
Run cargo build-bpf --bpf-sdk ./sdk again. It should work this time.

rollschild · April 26, 2022, 3:58pm

Hi,

Thank you so much for the detailed explanation. I’m still not able to build. I will outline everything I did here.

I git cloned a brand new solana example-helloworld project and cd into this project.
Created a flake.nix using exactly what you provided and git add flake.nix
Ran nix build and got the following error:

error: flake 'git+file:///home/rollschild/projects/example-helloworld' does not provide attribute 'packages.x86_64-linux.default' or 'defaultPackage.x86_64-linux'

I added this line to the flake.nix and built again and succeeded building:

        defaultPackage = solana-cli-pkg;

Ran nix develop and entered the shell
Did a rustup toolchain list and this is the output:

[rollschild@nixos:~/projects/example-helloworld]$ rustup toolchain list
nightly-x86_64-unknown-linux-gnu (default)
bpf

Ran rustup toolchain uninstall bpf
Under the rust project directory, created sdk/ using mkdir
Ran cargo build-bpf --bpf-sdk ./sdk and got the cargo command not found error:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ cargo build-bpf --bpf-sdk ./sdk
BPF SDK: /home/rollschild/projects/example-helloworld/src/program-rust/sdk
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: rustup toolchain link bpf /home/rollschild/projects/example-helloworld/src/program-rust/sdk/dependencies/bpf-tools/rust
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
error: command failed: 'cargo': No such file or directory (os error 2)

I did a which cargo and the output was /nix/store/rzb0h2kzbl09q5z8z6d98m5b639pihl1-rustup-1.24.3/bin/cargo, so looks like it was using rustup’s binary for cargo, not the standalone cargo binary
At this point I didn’t know whether I should handle the cargo not found error or run autopatchelf so I just did autopatchelf:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ../../../nix-patchtools/result/bin/autopatchelf ~/.cache/solana/
Patching all ELF binaries in directory: /home/rollschild/.cache/solana/
Auto patching ELF binary: /home/rollschild/.cache/solana/v1.23/bpf-tools/llvm/bin/llvm-readobj
No package found that provides library: libpthread.so.0

Looks like some libraries were missing, which prevented patching. I don’t know how to fix this to be honest.
I added cargo to the flake.nix and re-built and re-nix develop

      devShell = mkShell {
        buildInputs = [
          rust-analyzer
          solana-cli-pkg 
          solana-bpf-tools-pkg
          cargo-edit
          cargo # ADDED HERE
          rustup
        ];

Now if I ran cargo build-bpf --bpf-sdk ./sdk again I got the classic bpf error:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ cargo build-bpf --bpf-sdk ./sdk
BPF SDK: /home/rollschild/projects/example-helloworld/src/program-rust/sdk
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: rustup toolchain link bpf /home/rollschild/projects/example-helloworld/src/program-rust/sdk/dependencies/bpf-tools/rust
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
error: no such subcommand: `+bpf`

        Did you mean `b`?

I tried patching again using autopatchelf but got the same No package found that provides library: libpthread.so.0 error
So one of the questions is, should I install/use the standalone cargo binary or use the one provided by rustup? They both failed, as listed above but provided two different kinds of errors.

One note is that the ./sdk/dependencies/bpf-tools is correctly symlinked:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ls -l ./sdk/dependencies/
total 0
lrwxrwxrwx 1 rollschild users 46 Apr 26 11:54 bpf-tools -> /home/rollschild/.cache/solana/v1.23/bpf-tools

itsfarseen · April 26, 2022, 4:13pm

Hi,
Good that you posted the whole thing, I will test it out and get back to you by tomorrow.

Meanwhile,

nix build won’t work with the flake file I posted. Only nix develop will work.
cargo from rustup should be used, not the standalone cargo.

Regards,
Farseen

rollschild · April 26, 2022, 4:54pm

Now that bpf-tools 1.25 is out, I modified its version in flake.nix, 1.23 → 1.25 and re-nix-develop and went through all the steps. However, I was not able to see 1.25 at all. I’m still seeing v1.23 and ./sdk/dependencies/bpf-tools is still symlinked to ~/.cache/solana/v1.23, which is very weird since I had already removed 1.23 everywhere:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ls -l sdk/dependencies/
total 0
lrwxrwxrwx 1 rollschild users 46 Apr 26 12:45 bpf-tools -> /home/rollschild/.cache/solana/v1.23/bpf-tools

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ls ~/.cache/solana/
v1.23

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ cat ../../flake.nix | grep version
        name = "solana-${version}";
        version = "1.10.3";
          url = "https://github.com/solana-labs/solana/releases/download/v${version}/${filename}";
        name = "solana-bpf-tools-${version}";
        version = "1.25"; # It's 1.25, no 1.23 at all
          url = "https://github.com/solana-labs/bpf-tools/releases/download/v${version}/solana-bpf-tools-linux.tar.bz2";

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$

itsfarseen · April 26, 2022, 5:08pm

Ooh that happens because of how Nix invalidates the store.

Basically, even if you change the URL and keep the sha, it will look at the sha and the sha in the store and decide it has not changed. It won’t download the URL and see if its sha matches with the sha written in the flake.

So, just remove the sha to make it re-download.
Then it will show that sha doesn’t match and show the correct sha. Copy that sha into the flake.nix.

rollschild · April 26, 2022, 5:21pm

nix build won’t work with the flake file I posted. Only nix develop will work.

Thanks for reminding me that. I will just do nix develop from now on.

cargo from rustup should be used, not the standalone cargo.

Yeah that’s what I thought but my stupid self couldn’t get cargo working without explicitly putting it into buildInputs. I ran <path-to-autopatchelf>/autopatchelf ~/.rustup/toolchains/bpf but it didn’t help.

Thank you so much for troubleshooting this for me.

rollschild · April 26, 2022, 5:22pm

So, just remove the sha to make it re-download.
Then it will show that sha doesn’t match and show the correct sha. Copy that sha into the flake.nix.

Thanks!!! I will try this.

itsfarseen · April 27, 2022, 9:41am

I re-ran the whole thing and this is what was missing:

libs="/home/farseen/.cache/solana/v1.23/bpf-tools/rust/lib:/nix/store/18fz9jnhmfkzkh6p1iwwwng4i7x4rag7-gcc-10.3.0-lib/lib:/nix/store/9b9ryxskcwh573jwjz6m5l01whkcb39a-zlib-1.2.11/lib:/nix/store/6w8g7njm4mck5dmjxws0z1xnrxvl81xa-glibc-2.34-115/lib:/nix/store/6pcq8w0hnqi9ypd0fl22xhwlxjb6pvir-openssl-1.1.1n/lib" ~/nix-patchtools/result/bin/autopatchelf ./

This is how you should have run autopatchelf. Sorry that I missed the libs= part.

Copy-pasting the exact command won’t work though, as the hashes could be different on your system.

You need:

/home/<username>/.cache/solana/v1.23/bpf-tools/rust/lib
/nix/store/<hash>-gcc-10.x.y/lib
/nix/store/<hash>-zlib-1.x.y/lib
/nix/store/<hash>-glibc-2.34-y/lib
/nix/store/<hash>-openssl-1.1.1n/lib

cd /nix/store and ls | grep <library-name> to find the actual paths.
Then concatenate them with ‘:’ as separator, put it in libs env var.

Then run autopatchelf as before.

rollschild · April 28, 2022, 1:30am

Build succeeded. Thanks a lot!!!

itsfarseen · May 15, 2022, 6:00pm

I finally put everything I did into a nix flake: GitHub - itsfarseen/solana-flake: Nix flake for Solana development

In addition, I patched cargo build-bpf to not download BPF Tools to ~/.cache and use the one provided by the flake instead. Because otherwise we’d have to run autopatchelf ourselves on ~/.cache/solana.

eureka-cpu · October 5, 2024, 8:41am

Hello @itsfarseen and @rollschild I know this is a bit of an old thread, but I’m currently running into some similar issues. I’m working off of this branch: Add `simple` e2e test; package `solana-cli:v1.18.22` and scripts by eureka-cpu · Pull Request #20 · anagrambuild/bonsol · GitHub

Currently I have cargo-build-sbf showing that the nix store sdk is linked, I’ve used autopatchelf and taken hints from @itsfarseen 's repository as well as @cideM 's saber repo and the solana packages on nixpkgs. It feels like it’s really close to working. I would love it if I didn’t need to use rustup to get this working but I’m currently at this error even with the sdk linked, rustup as a buildInputs etc:

$ cargo build-sbf                                                                                                                                  [1:24:17]
error: no such command: `+solana`

	Cargo does not handle `+toolchain` directives.
	Did you mean to invoke `cargo` through `rustup` instead?
FAIL

Is there anything that looks like it’s standing out as inherently wrong? Anything I’m missing? Any help is appreciated greatly <3

eureka-cpu · October 6, 2024, 8:01pm

I was able to get it working by creating a shell with mkShell instead of craneLib.devShell which uses fenix for the rust toolchain. Now I’m just trying to figure out if I can either just use fenix or if I have to use rustup and somehow make rustup aware of the fenix toolchain.

reichswanderer · March 11, 2025, 11:11pm

Given that I’ve run into the same issue, and I don’t use Flakes, it’s just one of the more difficult things on using nixos, honestly

eureka-cpu · March 12, 2025, 1:07am

I have a working flake that I am going to upstream into nixpkgs, I’ll respond here with the pull request link soon

reichswanderer · March 25, 2025, 4:56pm

I would really appreciate if you could share this flake, thank you very much!