PLCT Lab intends to collaborate on RISC-V support

[dramforever]

I am dram, currently working at PLCT Lab at ISCAS (Institute of Software, Chinese Academy of Sciences). We’ve been working with various RISC-V software and hardware communities to improve the ecosystem.

As part of Project Jiachen, also known as RISC-V Prosperity 2036, PLCT Lab intends to collaborate with the NixOS community. The goal of this collaboration is to improve Nixpkgs/NixOS support of the RISC-V architecture, thereby promoting the growth and prosperity of the RISC-V ecosystem.

To achieve this, we plan to provide access to RISC-V development boards to the NixOS community for use as CI and manual testing platforms. Additionally, we will offer internship opportunities (currently limited to Chinese students) to work on maintaining the CI infrastructure and fixing build failures.

We have already sponsored @NickCao with a build server that powers https://hydra.nichi.co, which builds Nixpkgs cross compiled to RISC-V from x86_64, and has been able to assist the community by catching and fixing cross problems and providing a RISC-V binary cache.

We look forward to working together with the NixOS community to drive innovation and progress in the RISC-V landscape.

Currently we would like to discuss the following questions with other community members:

1. What are your current views on RISC-V support? Is this a project worth pursuing?
2. What level of extra burden will this place on maintainers and infrastructure (edited to clarify) to gradually raise RISC-V platforms to better tiers of support? What can we do to better support for collaborators?
3. What approach should we go for? Should we prioritize cross-compilation or native compilation for RISC-V?
4. What are some difficulties and hurdles expected?

If you have any thoughts or opinions, just write them here as a reply. You can also contact me on Matrix in Exotic Nix Targets (#exotic:nixos.org) or NixOS RISC-V (#riscv:nixos.org) (I’m @dramforever:matrix.org). Thanks for your attention and support.

[onny]

Great to hear of the collaboration effort
Would like to see support of RiscV64 as official supported architecture in nixpkgs so no external binary cache would be required.

Already wrote a wiki page for NixOS on RISCV/VisionFive 2 - NixOS Wiki and looking forward mainline Linux kernel support for that device

[doronbehar]

@dramforever

What are your current views on RISC-V support? Is this a project worth pursuing?

The idea of cross compiling Nixpkgs to other embedded-like architectures is pretty cool, but from my experience with embedded Linux boards, armv7l is a much more common architecture. I have a personal project that uses the Balena ecosystem, and I deeply wish that some day I’d be able to fully build and deploy my project using Nix and Nixpkgs’ cross compilation support. I talked with the Balena folks once about supplying Nix support, but it is really a totally different product that I was proposing them, so I don’t think they’ll put any efforts in to this direction.

Unfortunately now, such a build and deployment flow would require me to cross compile packages like xorg and chromium, and I can’t afford to do that. If there was a hydra instance that would cross compile cache for armv7, and I could have known what is the Nixpkgs revision that this hydra instance has last evaluated, it would have been amazing. Such a cache would have also motivated me to help you in the mission of making Nixpkgs more cross compile-able.

@dramforever

What level of extra burden will this place on maintainers and infrastructure? What can we do to better support for collaborators?

Are you talking about Nixpkgs’ packages’ maintainers? If from time to time you’d send PRs to Nixpkgs that would fix cross compilation support for packages, no maintainer would object.

Also what infrastructure burden are you talking about? I think I didn’t understand the question.

@dramforever

What approach should we go for? Should we prioritize cross-compilation or native compilation for RISC-V

In my experience, if you fix a cross compilation issue to 1 platform, you fix it for all of them. So, yes! I support prioritize cross compilation!

@dramforever

What are some difficulties and hurdles expected?

I anticipate that you’d experience difficulty with the scopes in Nixpkgs, and with packages that need to mix packages from different scopes such as python3Packages and qt6Packages. For example picard, has this in it’s arguments (roughly quoting):

{ lib,
# ...
, python311Packages
, qt5
, gst_all_1
}:

Similarly, a common issue / mistake is packages that use the darwin scope in their arguments.

The classic way to support cross compilation of such packages is to callPackage them with inherit (<scope>) p1 p2 p3;…

Thinking about this common issue broadly, makes me think that a significant improvement to the pkgsCross overlays would be to support packages files that use such scope arguments. This would also allow to reduce all-packages.nix even further beyond what pkgs/by-name allows.

Another common issue / mistake related to scopes I suspect you’d encounter is with packages that are defined in all-packages.nix using <scope>.callPackage (e.g python3Packages.callPackage). I don’t know exactly how to explain why these will fail to cross compile, but I’m pretty sure they will.

[thoth]

I just wanted to chime in and say this is very exciting. I think RISC-V and it’s underlying licensing model is vastly superior to what is going on with ARM at the moment. Here’s to hoping we see more hardware soon.

I keep looking in the pine64 store to see if their tablet is available for purchase.

PineTab-V

but it seems to not be available yet. And is the rvphone ever going to come out? Are there any devices out there in the wild with risc-v on it yet?

minor edit: just adding a link to the dev boards that are available out there in the wild. Hope to see more consumer devices soon.

[dramforever]

@doronbehar

Are you talking about Nixpkgs’ packages’ maintainers? If from time to time you’d send PRs to Nixpkgs that would fix cross compilation support for packages, no maintainer would object.

Also what infrastructure burden are you talking about? I think I didn’t understand the question.

Hi, thanks for your answer. I’m glad that you’re supportive of improving support for cross and exotic.

As for infrastructure, I meant to refer to the burden adding and maintaining RISC-V CI in a more official way, and eventually getting riscv{32,64}-* to climb the (RFC64) support tiers. This would require more commitment.

[doronbehar]

@dramforever

As for infrastructure, I meant to refer to the burden adding and maintaining RISC-V CI in a more official way, and eventually getting riscv{32,64}-* to climb the (RFC64) support tiers. This would require more commitment.

Oh I see. Well, I don’t have a strong opinion about this since I’m not involved in the Hydra maintenance:), but in principal I support adding more packages to release-cross.nix. See also discussion here:

RFC: Extend cross compilation support in Hydra?

Lately I’ve been playing a lot with cross compilation. Indeed it feels like a 1st class feature of Nixpkgs! I was wondering: Every time there’s a staging-next -> master merge, I find myself cross compiling bash and other common packages which almost always cross compile successfully. Do you think that NixOS will be interested in extending the list of pkgsCross... attributes that hydra builds by default? I think this can beneficial for advertising this feature as well. Linking armv7 Hydra channe…

[dramforever]

I see Pinetab-V here PineTab Archives - PINE STORE

There’s also Muse Book, and DC ROMA laptops. And as you mentioned, loads of dev boards.

[thoth]

ya but the links only say ‘read more’ I can’t find a buy button for the risc-v tablets, only the arm tablets appear to be available at this time.

Thanks for pointing out the muse book, which looks to be available here.

[Atemu]

@dramforever

What are your current views on RISC-V support? Is this a project worth pursuing?

Absolutely.

@dramforever

What level of extra burden will this place on maintainers and infrastructure (edited to clarify) to gradually raise RISC-V platforms to better tiers of support? What can we do to better support for collaborators?

I doubt it’s any more of an issue than aarch64-linux and I never had major issues with that platform as a maintainer. Certainly less of an issue than Darwin.

@dramforever

What approach should we go for? Should we prioritize cross-compilation or native compilation for RISC-V?

Given there isn’t really much hardware that is reasonably capable of compiling software and common usage in embedded systems, I’d prioritise cross-compilation.

[Mic92]

I would be cool to have an riscv64 builder in nix-community so that we start testing all our projects on riscv64 as well: GitHub - nix-community/infra: nix-community infrastructure [maintainer=@zowoq]

[Mic92]

We also have machines were we give people from the community access to test out interactively. This would also help to improve riscv64 support.

[dramforever]

Currently IIUC most riscv64 work is done with cross or qemu-user-binfmt which doesn’t need a riscv64 machine, because so far riscv64 machines have honestly been disappointing in performance. There’s also no KVM-capable hardware to run NixOS tests on.

For interactive testing PLCT Lab can provide access to real riscv hardware at https://rvlab.org/ subject to availability. If root/hardware access is needed for NixOS/kernel/bootloader stuff it’s negotiable but I’ll have to verify what infrastructure we have (Remote reset button poking, UART console, JTAG …)

[chuangzhu]

What approach should we go for? Should we prioritize cross-compilation or native compilation for RISC-V?

Cross compilation works around the bootstrapping problems and make many things appear to work. But if RISC-V is targeting any other use cases than embedded systems, it’s unavoidable to make native compilations work

[chuangzhu]

About the performance issue of current RISC-V machines, I heard that PLCT Lab got some 64-core 2GHz Milk-V Pioneer builders for Arch Linux development. I wonder how they actually performed. Single core performances may not be that important since Nix builds are well parallelized

[Atemu]

@chuangzhu

Single core performances may not be that important since Nix builds are well parallelized

IME single core perf is almost always in the critical path for a significant portion of each build.

[Mic92]

@dramforever If @NickCao is operating now a Hydra using cross-compilation, maybe we should put his hydra cache in the wiki? RISC-V - NixOS Wiki

[doronbehar]

A related cross compilation active thread that may interest people here:

Frustrations about splicing

I made splicing (nativeDrv and crossDrv are older, but were done differently). Splicing is bad. I never wanted it. I causes major headaches when getting anywhere near bootstrapping. It is gross. It is probably slow. It should be removed. https://github.com/NixOS/nixpkgs/issues/204303#issuecomment-1383282367 Splicing is too magical. Probably three people really understand it. Dozens of people think they understand it but really don’t, and get pissed off when it breaks in weird ways. Th…

[AndersonTorres]

I am answering mostly as a fanatic enthusiast of RISC-V:

@dramforever

Currently we would like to discuss the following questions with other community members:

1. What are your current views on RISC-V support? Is this a project worth pursuing?

As an enthusiast of this architecture since I abandoned undergrad, my answer is most certainly positive.
In an ideal world I would easily use a RISC-V powered notebook, or even some sort of mini-PC like a Beelink!

2. What level of extra burden will this place on maintainers and infrastructure (edited to clarify) to gradually raise RISC-V platforms to better tiers of support? What can we do to better support for collaborators?

As Doronbehar said, the people from Hydra and ofBorg can answer this.
But, again, I full support this! If we keep a mostly closed platform (Darwin), a mostly open should be easier.
Let’s just create a team!

3. What approach should we go for? Should we prioritize cross-compilation or native compilation for RISC-V?

In practical terms, nowadays there is no RISC-V machine powerful enough to native-compile RISC-V software. Unfortunately, cross-compilation is the most feasible road right now.

Nonetheless, with the advent of moderate RISC-V machines, I would like a bit of native compilation.

Talking a bit from my point of view, I want to build a bootstrap seed project for RISC-V, similar to GNU MES project. Motivated by the infamous Ken Thompson’s ROTT lecture, native compilation is mandatory

4. What are some difficulties and hurdles expected?

Nixpkgs

Seriously speaking, Nixpkgs has a long but promising road.
In my megalomaniac dream, Nixpkgs should strive to be as portable as NetBSD.

@dramforever

There’s also Muse Book, and DC ROMA laptops. And as you mentioned, loads of dev boards.

From my peculiar point of view, they are still too expensive.
With the same amount of money I can buy a so much better x64 laptop…

[NickCao]

A random status update: GNOME (with many components disabled) is working!