Any tutorials/guides on Solana development?

[rollschild]

oh and in his PR that I linked above, there is a screenshot showing that he successfully ran cargo build-bpf, which I’m not sure how. Because it would give me the “read-only file system” error.

[itsfarseen]

Because it would give me the “read-only file system” error.

Their PR patches the cargo-build-bpf binary to not download the sdk to /nix/store/<some-folder>. That’s how they are able to do that.

I’m not really sure how to run this patchelf , since I would need to run this command on executables, not directories, am I right?

I’m so sorry I messed up this bit. It’s not patchelf, it’s autopatchelf from here: GitHub - svanderburg/nix-patchtools: Autopatching binary packages to make them work with Nix

I forgot which one I used but I remembered when you mentioned directories.

I think it’s enough to run autopatchelf on ./sdk, since the toolchain is symlinked to there.

[rollschild]

Okay, I’ve sorta made some progress?

I slightly modified my shellHook to set the $BPF_SDK_PATH:

          shellHook = ''
            mkdir -p ~/projects/example-helloworld/src/program-rust/sdk
            export BPF_SDK_PATH="/home/rollschild/projects/example-helloworld/src/program-rust/sdk"
            export LIBCLANG_PATH="${llvmPackages_7.libclang.lib}/lib"

            rm -rf ~/.cache/solana*
            mkdir -p ~/.cache/solana/${solana-bpf-tools-pkg.version}
            ln -s ~/projects/example-helloworld/src/program-rust/sdk/dependencies/bpf-tools ~/.cache/solana/${solana-bpf-tools-pkg.version}/bpf-tools
          '';

so that cargo build-bpf no longer fails on the “readonly file system” error.

But I’m still seeing bpf no subcommand errors.

So I git cloned the autopatchelf repo and ran nix-build to build autopatchelf into the nix-patchtools/result/bin/autopatchelf. Then I in my current project I ran this command but got the No package found that provides library: librustc_driver-a8aaff3fbd0bfff2.so error:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ../../../nix-patchtools/result/bin/autopatchelf sdk/dependencies/bpf-tools/rust/
Patching all ELF binaries in directory: sdk/dependencies/bpf-tools/rust/
Auto patching ELF binary: sdk/dependencies/bpf-tools/rust/bin/rustc
No package found that provides library: librustc_driver-a8aaff3fbd0bfff2.so

which seems weird, because sdk/dependencies/bpf-tools is just a symlink to ~/.cache/solana/v1.23/bpf-tools, which looks correct…

[itsfarseen]

I’ll give you the cause of each error messages:

• bpf no subcommand:
  cargo doesn’t have a +bpf subcommand. +<toolchain> is a rustup subcommand.
  Non-rustup cargo is overriding the rustup cargo.
• command not found: The binary that is being called has an invalid loader path. Fix it using autopatchelf.
• read only filesystem: Solana sdk is trying to download the bpf sdk into /nix/store/.
  Fix it by giving an explicit --bpf-sdk path.

I had messed up a bunch of things like forgetting about autopatchelf when I gave you the initial instructions. But now that I have given you everything, you should be able to use the flake.nix I shared in the first post and get it working.

[itsfarseen]

@itsfarseen

{
  description = "";
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs";
    rust-overlay.url = "github:oxalica/rust-overlay";
    flake-utils.url  = "github:numtide/flake-utils";
  };
  outputs = { self, nixpkgs, rust-overlay, flake-utils, ... }: flake-utils.lib.eachDefaultSystem (system:
    let
      overlays = [ (import rust-overlay) ];
      pkgs = import nixpkgs {
        inherit system overlays;
      };
      solana-cli = {stdenv, fetchurl, fetchzip, lib, autoPatchelfHook, pkgs }: stdenv.mkDerivation rec {
        name = "solana-${version}";
        version = "1.10.3";
        filename = "solana-release-x86_64-unknown-linux-gnu.tar.bz2";
        src = fetchzip {
          url = "https://github.com/solana-labs/solana/releases/download/v${version}/${filename}";
          sha256 = "sha256-jOUCyK7spTeEZz+h2hxfGpDkZ+2pYbyUFzlQ20c1zv4=";
        };
        nativeBuildInputs = [ autoPatchelfHook pkgs.makeWrapper ];
        buildInputs = with pkgs; [
          sgx-sdk
          ocl-icd
          eudev
          rustup
          stdenv.cc.cc
        ];

        installPhase = ''
          mkdir -p $out;
          cp -r bin $out;
          mkdir -p $out/bin/sdk/;
          ln -s "${solana-bpf-tools-pkg}" $out/bin/sdk/bpf;
          chmod 0755 -R $out;
        '';

        meta = with lib; {
          homepage = "https://docs.solana.com/cli/install-solana-cli-tools#download-prebuilt-binaries";
          platforms = platforms.linux;
        };
      };

      solana-bpf-tools = {stdenv, fetchurl, fetchzip, lib, autoPatchelfHook, pkgs }: stdenv.mkDerivation rec {
        name = "solana-bpf-tools-${version}";
        version = "1.23";
        src = fetchzip {
          url = "https://github.com/solana-labs/bpf-tools/releases/download/v${version}/solana-bpf-tools-linux.tar.bz2";
          sha256 = "sha256-4aWBOAOcGviwJ7znGaHbB1ngNzdXqlfDX8gbZtdV1aA=";
          stripRoot = false;
        };

        nativeBuildInputs = [ autoPatchelfHook ];
        buildInputs = with pkgs; [
          zlib
          stdenv.cc.cc
          openssl
        ];

        installPhase = ''
          mkdir -p $out;
          cp -r $src/llvm $out;
          cp -r $src/rust $out;
          chmod 0755 -R $out;
        '';

        meta = with lib; {
          homepage = "https://github.com/solana-labs/bpf-tools/releases";
          platforms = platforms.linux;
        };
      };
      solana-cli-pkg = (pkgs.callPackage solana-cli {});
      solana-bpf-tools-pkg = (pkgs.callPackage solana-bpf-tools {});
    in with pkgs; {
      devShell = mkShell {
        buildInputs = [
          rust-analyzer
          solana-cli-pkg 
          solana-bpf-tools-pkg
          cargo-edit
          rustup
        ];
      };
    }
  );
}

To recap:

• Use the above flake.nix. It will make rustup and cargo build-bpf available.
• Install a normal rust toolchain using rustup. Skip this if you already have installed. Run rustup toolchain list to see.
• Remove bpf toolchain if it is shown in the list, to start from a blank slate. rustup toolchain uninstall bpf.
• Run cargo build-bpf --bpf-sdk ./sdk. It will download bpf sdk to ./sdk/dependencies.
  After downloading dependencies, it will try to run it, but it will fail, because we haven’t run patchelf on it. This is normal.
• Run autopatchelf ~/.cache/solana. The stuff that got downloaded in ./sdk is actually downloaded to ~/.cache/solana and symlinked from there.
• Symlink the following two things to ./sdk
  • ln -s /nix/store/c11gzzi5ax9b8kmqrizrazjil4m8c4dl-solana-1.10.3/bin/sdk/bpf/env.sh ./sdk/
  • ln -s /nix/store/c11gzzi5ax9b8kmqrizrazjil4m8c4dl-solana-1.10.3/bin/sdk/bpf/scripts ./sdk/
    The hash might be different in your system. Run fd or find or ls | grep in /nix/store to find the solana folder and substitute it’s path in place of c11gzz...c4dl-solana-1.10.3
• Run cargo build-bpf --bpf-sdk ./sdk again. It should work this time.

[rollschild]

Hi,

Thank you so much for the detailed explanation. I’m still not able to build. I will outline everything I did here.

1. I git cloned a brand new solana example-helloworld project and cd into this project.
2. Created a flake.nix using exactly what you provided and git add flake.nix
3. Ran nix build and got the following error:

error: flake 'git+file:///home/rollschild/projects/example-helloworld' does not provide attribute 'packages.x86_64-linux.default' or 'defaultPackage.x86_64-linux'

4. I added this line to the flake.nix and built again and succeeded building:

        defaultPackage = solana-cli-pkg;

5. Ran nix develop and entered the shell
6. Did a rustup toolchain list and this is the output:

[rollschild@nixos:~/projects/example-helloworld]$ rustup toolchain list
nightly-x86_64-unknown-linux-gnu (default)
bpf

7. Ran rustup toolchain uninstall bpf
8. Under the rust project directory, created sdk/ using mkdir
9. Ran cargo build-bpf --bpf-sdk ./sdk and got the cargo command not found error:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ cargo build-bpf --bpf-sdk ./sdk
BPF SDK: /home/rollschild/projects/example-helloworld/src/program-rust/sdk
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: rustup toolchain link bpf /home/rollschild/projects/example-helloworld/src/program-rust/sdk/dependencies/bpf-tools/rust
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
error: command failed: 'cargo': No such file or directory (os error 2)

10. I did a which cargo and the output was /nix/store/rzb0h2kzbl09q5z8z6d98m5b639pihl1-rustup-1.24.3/bin/cargo, so looks like it was using rustup’s binary for cargo, not the standalone cargo binary
11. At this point I didn’t know whether I should handle the cargo not found error or run autopatchelf so I just did autopatchelf:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ../../../nix-patchtools/result/bin/autopatchelf ~/.cache/solana/
Patching all ELF binaries in directory: /home/rollschild/.cache/solana/
Auto patching ELF binary: /home/rollschild/.cache/solana/v1.23/bpf-tools/llvm/bin/llvm-readobj
No package found that provides library: libpthread.so.0

12. Looks like some libraries were missing, which prevented patching. I don’t know how to fix this to be honest.
13. I added cargo to the flake.nix and re-built and re-nix develop

      devShell = mkShell {
        buildInputs = [
          rust-analyzer
          solana-cli-pkg 
          solana-bpf-tools-pkg
          cargo-edit
          cargo # ADDED HERE
          rustup
        ];

14. Now if I ran cargo build-bpf --bpf-sdk ./sdk again I got the classic bpf error:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ cargo build-bpf --bpf-sdk ./sdk
BPF SDK: /home/rollschild/projects/example-helloworld/src/program-rust/sdk
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: rustup toolchain link bpf /home/rollschild/projects/example-helloworld/src/program-rust/sdk/dependencies/bpf-tools/rust
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
error: no such subcommand: `+bpf`

        Did you mean `b`?

15. I tried patching again using autopatchelf but got the same No package found that provides library: libpthread.so.0 error
16. So one of the questions is, should I install/use the standalone cargo binary or use the one provided by rustup? They both failed, as listed above but provided two different kinds of errors.

One note is that the ./sdk/dependencies/bpf-tools is correctly symlinked:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ls -l ./sdk/dependencies/
total 0
lrwxrwxrwx 1 rollschild users 46 Apr 26 11:54 bpf-tools -> /home/rollschild/.cache/solana/v1.23/bpf-tools

[itsfarseen]

Hi,
Good that you posted the whole thing, I will test it out and get back to you by tomorrow.

Meanwhile,

• nix build won’t work with the flake file I posted. Only nix develop will work.
• cargo from rustup should be used, not the standalone cargo.

Regards,
Farseen

[rollschild]

Now that bpf-tools 1.25 is out, I modified its version in flake.nix, 1.23 → 1.25 and re-nix-develop and went through all the steps. However, I was not able to see 1.25 at all. I’m still seeing v1.23 and ./sdk/dependencies/bpf-tools is still symlinked to ~/.cache/solana/v1.23, which is very weird since I had already removed 1.23 everywhere:

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ls -l sdk/dependencies/
total 0
lrwxrwxrwx 1 rollschild users 46 Apr 26 12:45 bpf-tools -> /home/rollschild/.cache/solana/v1.23/bpf-tools

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ls ~/.cache/solana/
v1.23

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ cat ../../flake.nix | grep version
        name = "solana-${version}";
        version = "1.10.3";
          url = "https://github.com/solana-labs/solana/releases/download/v${version}/${filename}";
        name = "solana-bpf-tools-${version}";
        version = "1.25"; # It's 1.25, no 1.23 at all
          url = "https://github.com/solana-labs/bpf-tools/releases/download/v${version}/solana-bpf-tools-linux.tar.bz2";

[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ 

[itsfarseen]

Ooh that happens because of how Nix invalidates the store.

Basically, even if you change the URL and keep the sha, it will look at the sha and the sha in the store and decide it has not changed. It won’t download the URL and see if its sha matches with the sha written in the flake.

So, just remove the sha to make it re-download.
Then it will show that sha doesn’t match and show the correct sha. Copy that sha into the flake.nix.

[rollschild]

• nix build won’t work with the flake file I posted. Only nix develop will work.

Thanks for reminding me that. I will just do nix develop from now on.

• cargo from rustup should be used, not the standalone cargo.

Yeah that’s what I thought but my stupid self couldn’t get cargo working without explicitly putting it into buildInputs. I ran <path-to-autopatchelf>/autopatchelf ~/.rustup/toolchains/bpf but it didn’t help.

Thank you so much for troubleshooting this for me.

[rollschild]

So, just remove the sha to make it re-download.
Then it will show that sha doesn’t match and show the correct sha. Copy that sha into the flake.nix.

Thanks!!! I will try this.

[itsfarseen]

I re-ran the whole thing and this is what was missing:

libs="/home/farseen/.cache/solana/v1.23/bpf-tools/rust/lib:/nix/store/18fz9jnhmfkzkh6p1iwwwng4i7x4rag7-gcc-10.3.0-lib/lib:/nix/store/9b9ryxskcwh573jwjz6m5l01whkcb39a-zlib-1.2.11/lib:/nix/store/6w8g7njm4mck5dmjxws0z1xnrxvl81xa-glibc-2.34-115/lib:/nix/store/6pcq8w0hnqi9ypd0fl22xhwlxjb6pvir-openssl-1.1.1n/lib" ~/nix-patchtools/result/bin/autopatchelf ./

This is how you should have run autopatchelf. Sorry that I missed the libs= part.

Copy-pasting the exact command won’t work though, as the hashes could be different on your system.

You need:

• /home/<username>/.cache/solana/v1.23/bpf-tools/rust/lib
• /nix/store/<hash>-gcc-10.x.y/lib
• /nix/store/<hash>-zlib-1.x.y/lib
• /nix/store/<hash>-glibc-2.34-y/lib
• /nix/store/<hash>-openssl-1.1.1n/lib

cd /nix/store and ls | grep <library-name> to find the actual paths.
Then concatenate them with ‘:’ as separator, put it in libs env var.

Then run autopatchelf as before.

[rollschild]

Build succeeded. Thanks a lot!!!

[itsfarseen]

I finally put everything I did into a nix flake: GitHub - itsfarseen/solana-flake: Nix flake for Solana development

In addition, I patched cargo build-bpf to not download BPF Tools to ~/.cache and use the one provided by the flake instead. Because otherwise we’d have to run autopatchelf ourselves on ~/.cache/solana.