oh and in his PR that I linked above, there is a screenshot showing that he successfully ran cargo build-bpf
, which I’m not sure how. Because it would give me the “read-only file system” error.
Because it would give me the “read-only file system” error.
Their PR patches the cargo-build-bpf binary to not download the sdk to /nix/store/<some-folder>
. That’s how they are able to do that.
I’m not really sure how to run this
patchelf
, since I would need to run this command on executables, not directories, am I right?
I’m so sorry I messed up this bit. It’s not patchelf
, it’s autopatchelf
from here: GitHub - svanderburg/nix-patchtools: Autopatching binary packages to make them work with Nix
I forgot which one I used but I remembered when you mentioned directories.
I think it’s enough to run autopatchelf
on ./sdk
, since the toolchain is symlinked to there.
Okay, I’ve sorta made some progress?
I slightly modified my shellHook
to set the $BPF_SDK_PATH
:
shellHook = ''
mkdir -p ~/projects/example-helloworld/src/program-rust/sdk
export BPF_SDK_PATH="/home/rollschild/projects/example-helloworld/src/program-rust/sdk"
export LIBCLANG_PATH="${llvmPackages_7.libclang.lib}/lib"
rm -rf ~/.cache/solana*
mkdir -p ~/.cache/solana/${solana-bpf-tools-pkg.version}
ln -s ~/projects/example-helloworld/src/program-rust/sdk/dependencies/bpf-tools ~/.cache/solana/${solana-bpf-tools-pkg.version}/bpf-tools
'';
so that cargo build-bpf
no longer fails on the “readonly file system” error.
But I’m still seeing bpf no subcommand
errors.
So I git cloned the autopatchelf
repo and ran nix-build
to build autopatchelf
into the nix-patchtools/result/bin/autopatchelf
. Then I in my current project I ran this command but got the No package found that provides library: librustc_driver-a8aaff3fbd0bfff2.so
error:
[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ../../../nix-patchtools/result/bin/autopatchelf sdk/dependencies/bpf-tools/rust/
Patching all ELF binaries in directory: sdk/dependencies/bpf-tools/rust/
Auto patching ELF binary: sdk/dependencies/bpf-tools/rust/bin/rustc
No package found that provides library: librustc_driver-a8aaff3fbd0bfff2.so
which seems weird, because sdk/dependencies/bpf-tools
is just a symlink to ~/.cache/solana/v1.23/bpf-tools
, which looks correct…
I’ll give you the cause of each error messages:
-
bpf no subcommand
:
cargo
doesn’t have a+bpf
subcommand.+<toolchain>
is arustup
subcommand.
Non-rustupcargo
is overriding therustup
cargo. -
command not found
: The binary that is being called has an invalid loader path. Fix it usingautopatchelf
. -
read only filesystem
: Solana sdk is trying to download the bpf sdk into/nix/store/
.
Fix it by giving an explicit--bpf-sdk
path.
I had messed up a bunch of things like forgetting about autopatchelf
when I gave you the initial instructions. But now that I have given you everything, you should be able to use the flake.nix
I shared in the first post and get it working.
To recap:
- Use the above
flake.nix
. It will makerustup
andcargo build-bpf
available. - Install a normal rust toolchain using
rustup
. Skip this if you already have installed. Runrustup toolchain list
to see. - Remove
bpf
toolchain if it is shown in the list, to start from a blank slate.rustup toolchain uninstall bpf
. - Run
cargo build-bpf --bpf-sdk ./sdk
. It will download bpf sdk to./sdk/dependencies
.
After downloading dependencies, it will try to run it, but it will fail, because we haven’t runpatchelf
on it. This is normal. - Run
autopatchelf ~/.cache/solana
. The stuff that got downloaded in./sdk
is actually downloaded to~/.cache/solana
and symlinked from there. - Symlink the following two things to
./sdk
ln -s /nix/store/c11gzzi5ax9b8kmqrizrazjil4m8c4dl-solana-1.10.3/bin/sdk/bpf/env.sh ./sdk/
ln -s /nix/store/c11gzzi5ax9b8kmqrizrazjil4m8c4dl-solana-1.10.3/bin/sdk/bpf/scripts ./sdk/
The hash might be different in your system. Runfd
orfind
orls | grep
in/nix/store
to find thesolana
folder and substitute it’s path in place ofc11gzz...c4dl-solana-1.10.3
- Run
cargo build-bpf --bpf-sdk ./sdk
again. It should work this time.
Hi,
Thank you so much for the detailed explanation. I’m still not able to build. I will outline everything I did here.
- I git cloned a brand new solana example-helloworld project and
cd
into this project. - Created a
flake.nix
using exactly what you provided andgit add flake.nix
- Ran
nix build
and got the following error:
error: flake 'git+file:///home/rollschild/projects/example-helloworld' does not provide attribute 'packages.x86_64-linux.default' or 'defaultPackage.x86_64-linux'
- I added this line to the
flake.nix
and built again and succeeded building:
defaultPackage = solana-cli-pkg;
- Ran
nix develop
and entered the shell - Did a
rustup toolchain list
and this is the output:
[rollschild@nixos:~/projects/example-helloworld]$ rustup toolchain list
nightly-x86_64-unknown-linux-gnu (default)
bpf
- Ran
rustup toolchain uninstall bpf
- Under the rust project directory, created
sdk/
usingmkdir
- Ran
cargo build-bpf --bpf-sdk ./sdk
and got thecargo command not found
error:
[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ cargo build-bpf --bpf-sdk ./sdk
BPF SDK: /home/rollschild/projects/example-helloworld/src/program-rust/sdk
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: rustup toolchain link bpf /home/rollschild/projects/example-helloworld/src/program-rust/sdk/dependencies/bpf-tools/rust
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
error: command failed: 'cargo': No such file or directory (os error 2)
- I did a
which cargo
and the output was/nix/store/rzb0h2kzbl09q5z8z6d98m5b639pihl1-rustup-1.24.3/bin/cargo
, so looks like it was usingrustup
’s binary for cargo, not the standalonecargo
binary - At this point I didn’t know whether I should handle the
cargo not found
error or runautopatchelf
so I just didautopatchelf
:
[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ../../../nix-patchtools/result/bin/autopatchelf ~/.cache/solana/
Patching all ELF binaries in directory: /home/rollschild/.cache/solana/
Auto patching ELF binary: /home/rollschild/.cache/solana/v1.23/bpf-tools/llvm/bin/llvm-readobj
No package found that provides library: libpthread.so.0
- Looks like some libraries were missing, which prevented patching. I don’t know how to fix this to be honest.
- I added
cargo
to theflake.nix
and re-built and re-nix develop
devShell = mkShell {
buildInputs = [
rust-analyzer
solana-cli-pkg
solana-bpf-tools-pkg
cargo-edit
cargo # ADDED HERE
rustup
];
- Now if I ran
cargo build-bpf --bpf-sdk ./sdk
again I got the classicbpf
error:
[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ cargo build-bpf --bpf-sdk ./sdk
BPF SDK: /home/rollschild/projects/example-helloworld/src/program-rust/sdk
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: rustup toolchain link bpf /home/rollschild/projects/example-helloworld/src/program-rust/sdk/dependencies/bpf-tools/rust
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
error: no such subcommand: `+bpf`
Did you mean `b`?
- I tried patching again using
autopatchelf
but got the sameNo package found that provides library: libpthread.so.0
error - So one of the questions is, should I install/use the standalone
cargo
binary or use the one provided byrustup
? They both failed, as listed above but provided two different kinds of errors.
One note is that the ./sdk/dependencies/bpf-tools
is correctly symlinked:
[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ls -l ./sdk/dependencies/
total 0
lrwxrwxrwx 1 rollschild users 46 Apr 26 11:54 bpf-tools -> /home/rollschild/.cache/solana/v1.23/bpf-tools
Hi,
Good that you posted the whole thing, I will test it out and get back to you by tomorrow.
Meanwhile,
- nix build won’t work with the flake file I posted. Only nix develop will work.
- cargo from rustup should be used, not the standalone cargo.
Regards,
Farseen
Now that bpf-tools
1.25 is out, I modified its version in flake.nix
, 1.23
→ 1.25
and re-nix-develop
and went through all the steps. However, I was not able to see 1.25 at all. I’m still seeing v1.23 and ./sdk/dependencies/bpf-tools
is still symlinked to ~/.cache/solana/v1.23
, which is very weird since I had already removed 1.23 everywhere:
[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ls -l sdk/dependencies/
total 0
lrwxrwxrwx 1 rollschild users 46 Apr 26 12:45 bpf-tools -> /home/rollschild/.cache/solana/v1.23/bpf-tools
[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ ls ~/.cache/solana/
v1.23
[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ cat ../../flake.nix | grep version
name = "solana-${version}";
version = "1.10.3";
url = "https://github.com/solana-labs/solana/releases/download/v${version}/${filename}";
name = "solana-bpf-tools-${version}";
version = "1.25"; # It's 1.25, no 1.23 at all
url = "https://github.com/solana-labs/bpf-tools/releases/download/v${version}/solana-bpf-tools-linux.tar.bz2";
[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$
Ooh that happens because of how Nix invalidates the store.
Basically, even if you change the URL and keep the sha, it will look at the sha and the sha in the store and decide it has not changed. It won’t download the URL and see if its sha matches with the sha written in the flake.
So, just remove the sha to make it re-download.
Then it will show that sha doesn’t match and show the correct sha. Copy that sha into the flake.nix.
- nix build won’t work with the flake file I posted. Only nix develop will work.
Thanks for reminding me that. I will just do nix develop
from now on.
- cargo from rustup should be used, not the standalone cargo.
Yeah that’s what I thought but my stupid self couldn’t get cargo
working without explicitly putting it into buildInputs
. I ran <path-to-autopatchelf>/autopatchelf ~/.rustup/toolchains/bpf
but it didn’t help.
Thank you so much for troubleshooting this for me.
So, just remove the sha to make it re-download.
Then it will show that sha doesn’t match and show the correct sha. Copy that sha into the flake.nix.
Thanks!!! I will try this.
I re-ran the whole thing and this is what was missing:
libs="/home/farseen/.cache/solana/v1.23/bpf-tools/rust/lib:/nix/store/18fz9jnhmfkzkh6p1iwwwng4i7x4rag7-gcc-10.3.0-lib/lib:/nix/store/9b9ryxskcwh573jwjz6m5l01whkcb39a-zlib-1.2.11/lib:/nix/store/6w8g7njm4mck5dmjxws0z1xnrxvl81xa-glibc-2.34-115/lib:/nix/store/6pcq8w0hnqi9ypd0fl22xhwlxjb6pvir-openssl-1.1.1n/lib" ~/nix-patchtools/result/bin/autopatchelf ./
This is how you should have run autopatchelf
. Sorry that I missed the libs=
part.
Copy-pasting the exact command won’t work though, as the hashes could be different on your system.
You need:
/home/<username>/.cache/solana/v1.23/bpf-tools/rust/lib
/nix/store/<hash>-gcc-10.x.y/lib
/nix/store/<hash>-zlib-1.x.y/lib
/nix/store/<hash>-glibc-2.34-y/lib
/nix/store/<hash>-openssl-1.1.1n/lib
cd /nix/store
and ls | grep <library-name>
to find the actual paths.
Then concatenate them with ‘:’ as separator, put it in libs
env var.
Then run autopatchelf
as before.
Build succeeded. Thanks a lot!!!
I finally put everything I did into a nix flake: GitHub - itsfarseen/solana-flake: Nix flake for Solana development
In addition, I patched cargo build-bpf
to not download BPF Tools to ~/.cache
and use the one provided by the flake instead. Because otherwise we’d have to run autopatchelf
ourselves on ~/.cache/solana
.
Hello @itsfarseen and @rollschild I know this is a bit of an old thread, but I’m currently running into some similar issues. I’m working off of this branch: Add `simple` e2e test; package `solana-cli:v1.18.22` and scripts by eureka-cpu · Pull Request #20 · anagrambuild/bonsol · GitHub
Currently I have cargo-build-sbf
showing that the nix store sdk is linked, I’ve used autopatchelf and taken hints from @itsfarseen 's repository as well as @cideM 's saber repo and the solana packages on nixpkgs. It feels like it’s really close to working. I would love it if I didn’t need to use rustup to get this working but I’m currently at this error even with the sdk linked, rustup as a buildInputs etc:
$ cargo build-sbf [1:24:17]
error: no such command: `+solana`
Cargo does not handle `+toolchain` directives.
Did you mean to invoke `cargo` through `rustup` instead?
FAIL
Is there anything that looks like it’s standing out as inherently wrong? Anything I’m missing? Any help is appreciated greatly <3
I was able to get it working by creating a shell with mkShell
instead of craneLib.devShell
which uses fenix
for the rust toolchain. Now I’m just trying to figure out if I can either just use fenix or if I have to use rustup and somehow make rustup aware of the fenix toolchain.
Given that I’ve run into the same issue, and I don’t use Flakes, it’s just one of the more difficult things on using nixos, honestly
I have a working flake that I am going to upstream into nixpkgs, I’ll respond here with the pull request link soon