Recently I wanted to give Solana a try and build something. But the experience of setting up the dev environment has been disappointing.
One of the first steps is to install solana CLI. I wanted to do this “the Nix way” so I searched for Nix packages about Solana and all I found was this solana-testnet, which is supposed to package the solana-cli. Although by installing this package the solana-cli command was successfully installed, some other commands like solana-keygen are missing.
I git pulled Solana’s hello-world repo and tried to build it but ran into another issue, about which I already posted an issue on their repo:
> cargo build-bpf --manifest-path=./src/program-rust/Cargo.toml --bpf-out-dir=dist/program
error: no such subcommand: `build-bpf`
So my question is, has anyone successfully got Solana up and running on their NixOS? Could someone please point me to some repos, guides, tutorials, etc? I did a quick internet search but didn’t find too much useful information.
Hi @rollschild, I think most Solana devs don’t use Nix tools, unlike Cardano devs.
At my work, I was actually discouraged from using Nix and they told they won’t check-in nix related stuff to the repo.
If you run Nix on top of a traditional distro, your best bet is to go the non-Nix route.
I run NixOS and here’s what I did:
Install rustup (either globally or in shell.nix/flake.nix).
The solana cli will add bpf toolchain to rustup. I don’t remember which command did this, I searched my history and couldn’t find.
cargo build-bpf, on first run, will auto download bpf-tools to ~/.cache. It then tries to symlink to <path-to-cli>/bin/sdk which in my case was in nix store. Since nix store is readonly, it failed. So I had to provide cargo build-bpf --bpf-sdk ./sdk.
After the download completes, the build-bpf command will fail because of rpath differences. I had to run autopatchelf ~/.cache/solana/<version>/bpf-tools to fix this.
Now you need to symlink scripts folder and env.sh from solana cli folder to ./sdk.
I’m attaching a the flake.nix I used to get solana cli.
It contains references to bpf-tools, but it’s ignored by solana. Solana will download it’s own. I didn’t bother removing it from flake.nix hoping someday I will make Solana use this bpf-tools.
I also tried using rust-overlay at first. Later realized, solana needs rustup so that it can add its bpf toolchain.
in nix shell when I tried those commands such as solana-install-init I got the following error: bash: /home/username/.local/share/solana/install/active_release/bin/solana-install-init: No such file or directory
I followed this to use zsh, however I got the same “command not found” error: solana-install-init: command not found
I’m sure that my $PATH is already updated with this path:
I actually went in this directory and those “executables” are indeed in there but I wasn’t able to run any of them. They just returned command not found error.
All binaries compiled on other Linux distros need to be patched to be able to run on NixOS.
This has something to do with dynamic linking and library resolution in NixOS.
Command not found error is usually due to this this. It’s having trouble finding the d
In your case, try running patchelf command on the folder containing your executables.
I forgot the flags, and I’m on mobile right now, so can’t look it up; but I think I have given an example in the previous post. See if that works.
@itsfarseen Thanks so much for the detailed explanation! Looks like currently my issue is, I don’t know how to add the bpf toolchain to rustup:
The solana cli will add bpf toolchain to rustup . I don’t remember which command did this, I searched my history and couldn’t find.
Because based on what you said, I would need to have the bpf toolchain, then run the cargo build-bpf in order to patchelf the commands to the /.cache/solana/<version>/bpf-tools.
I just don’t know how to have this bpf toolchain. Clearly installing solana using the recommended way wouldn’t work on NixOS. I tried to install the crate by doing
@itsfarseen sorry, one more question: how did you install Solana at the first place? via nixpkgs? or using the shell-curl command that’s recommended on Solana’s doc? Using the shell command failed for me but from what I’ve found this should the correct way and would get rid of all the errors. But I’m not sure how this translates to NixOS.
Using the flake.nix I shared and running nix develop.
I don’t know how to add the bpf toolchain to rustup.
Here’s what I think is happening:
Once inside the nix shell, you will have cargo-build-bpf in your path.
Cargo will simply redirect cargo build-bpf to cargo-build-bpf.
Everytime cargo-build-bpf is ran, it checks if the bpf toolchain is added in rustup. If it’s not, then it links it from the path you specify in --bpf-sdk.
I’m the author of the mentioned flake and I just wanted to point out that you can’t currently use the custom BPF toolchain with Nix tooling as far as I can tell. It’s just not implemented by any Rustup replacement.
The flake is also a pretty big experiment especially since I don’t use Solana myself. For now I removed solana-install-init from the flake since that command is essentially replaced by Nix here. I’ll also add something to the README to make the status of this repo clearer.
@itsfarseen Thank you so much for the steps. Using your flake.nix and your steps I did have some progress but now I’m stuck in the cargo build-bpf step.
I ran cargo build-bpf --bpf-sdk sdk and the first two steps succeeded, which verified the bpf toolchain, but it failed on the actual compiling phase, inside nix-shell using nix develop:
[rollschild@nixos:~/projects/example-helloworld/src/program-rust]$ cargo build-bpf --bpf-sdk ./sdk
BPF SDK: /home/rollschild/projects/example-helloworld/src/program-rust/sdk
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: rustup toolchain link bpf /home/rollschild/projects/example-helloworld/src/program-rust/sdk/dependencies/bpf-tools/rust
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
error: command failed: 'cargo': No such file or directory (os error 2)
Now this cargo command failed really puzzled me. Doing a which cargo clearly gave me the correct path: /nix/store/429wcfhqydl7v3yzcapc1gvkyx6ss5yp-rustup-1.24.3/bin/cargo, so I don’t quite know why this error came from.
Doing a cargo build succeeded under the same directory, which means there is no issue with cargo itself I guess. So I think the issue is cargo-build-bpf cannot seem to find the correct cargo command? I’m just thinking out loud here.
Sorry for the back-and-forth stupid questions. I really appreciate your help!
so here bpf is already symlinked to the local ./sdk. So my question is, I know patchelf sets the dynamic loader, but if bpf is already symlinked to ./sdk/dependencies/bpf-tools/rust, will patchelf make a difference at all? Plus, I’m not really sure how to run this patchelf, since I would need to run this command on executables, not directories, am I right?
Plus, I’m wondering, if it would be better that we put some of those commands into flake.nix, such as inside shellHook, so that we don’t lose it afterwards.
I tried to combine this shell nix with the flake you provided but basically got the same bpf subcommand not found error. But I posted here anyway for your reference.
I’m trying to run his shell nix standalone in another git repo, see if it makes a difference.