Done: reMarkable1 cross-compile toolchain for Nix

Have you tried compiling it statically yet? That is what I was planning for my reM1 but haven’t tried that yet. You mentioned it being an option in the OP.

Yes, compiling it statically works. IMO we should still try to dynamically link as much as possible with on-device libraries.

It seems that is using standard nix linker. If you look at the dynamicLinker if-then-else tree be sure to put remarkable condition before generic arm one. I did that with special isRemarkable flag.

However even with that my programs compile with linker pointed to nonsensical:

file /nix/store/xrm21bz3p2na8yxckg4g89b5kcf754jy-hello-2.10-armv7l-unknown-linux-gnueabihf/bin/hello
/nix/store/xrm21bz3p2na8yxckg4g89b5kcf754jy-hello-2.10-armv7l-unknown-linux-gnueabihf/bin/hello: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /usr/lib/ld.so.1, for GNU/Linux 2.6.32, not stripped

I have no clue where it comes from.
Inspecting binutils-wrapper it seems that there is correct nix-support/dynamic-linker file with proper contents.

Yeah, I made the right change to the dynamic linker, see this commit. Do we need to override something else? For instance, using the proprietary toolchain, this can be done.

It seems that you forgot to set the flag in remarkable1 example system. Take a look at my example

I saw, that proprietary toolchain fix, but it does set the proper contents for nix-support/dynamic-linker file. Theoretically the same happens when we override dynamicLinker. For some reason tho its getting ignored?

Ah, right. I just set the flag now and got

root@reMarkable:~# file hello-2020-11-17
hello-2020-11-17: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /usr/lib/ld.so.1, for GNU/Linux 2.6.32, not stripped

So we have the same result, not really sure what to do with that linker and where it comes from.

Here are the only places in nixpkgs that contains lib.so.1. I also checked if the if-else tree was properly taking that branch by raising an error, and it was correct.

$ ag "ld.so.1"    
pkgs/development/compilers/fpc/mark-paths.patch
35:-  const defdynlinker='/lib/ld.so.1';
36:+  const defdynlinker='@dynlinker-prefix@/lib/ld.so.1';
55:-  const defdynlinker='/lib/ld.so.1';
56:+  const defdynlinker='@dynlinker-prefix@/lib/ld.so.1';
88:-  const defdynlinker='/lib/ld.so.1';
89:+  const defdynlinker='@dynlinker-prefix@/lib/ld.so.1';

pkgs/build-support/bintools-wrapper/default.nix
63:    else if targetPlatform.system == "powerpc-linux"  then "${libc_lib}/lib/ld.so.1"
64:    else if targetPlatform.isMips                     then "${libc_lib}/lib/ld.so.1"
66:    else if stdenv.lib.hasSuffix "pc-gnu" targetPlatform.config then "ld.so.1"

I think this might work;

But looking at the value for postFixup this doesn’t make sense, since the exact same logic is there already.

diff --git a/pkgs/build-support/bintools-wrapper/default.nix b/pkgs/build-support/bintools-wrapper/default.nix
index 06ae699d4ad..2f3e106bcd9 100644
--- a/pkgs/build-support/bintools-wrapper/default.nix
+++ b/pkgs/build-support/bintools-wrapper/default.nix
@@ -256,6 +256,11 @@ stdenv.mkDerivation {
       echo "-macosx_version_min 10.12 -sdk_version 10.12 -no_uuid" >> $out/nix-support/libc-ldflags-before
     ''
 
+    + optionalString targetPlatform.isRemarkable ''
+      echo "/lib/ld-linux-armhf.so.3" >$out/nix-support/dynamic-linker
+      echo "-dynamic-linker /lib/ld-linux-armhf.so.3" > $out/nix-support/libc-ldflags-before
+    ''
+
     ##
     ## User env support
     ##

if you inspect $out/nix-support/dynamic-linker correct value will be filled there already, since that is determined by this big if ... else dynamicLinker part

I think this value /usr/lib/ld.so.1 comes from binutils. I think that is the default set if the one passed as parameter is not found.

binutils source

Indeed, the thing we passed was invalid, so it must have fallen back on that.

In general, this patch is doing too much / trying to hard:

  • We already have generic logic getting the interpreter for 32-bit arm linux.
  • No need to patch anything if we don’t hit the default anyways
  • No need for prebuilt tools

All that remains is the zero-gravitas kernel config and example platform that uses it.

Initial implementation of remarkable1 cross-compile by siraben · Pull Request #102503 · NixOS/nixpkgs · GitHub has been merged! Huge thanks to @Ericson2314 for helping provide feedback. For any reMarkable users out there, here’s how to use it.

  1. Ensure that your nixpkgs channel is up to date (or equivalent with niv and flakes). You can check if it can cross-compile to reMarkable 1 by running
$ nix eval -f '<nixpkgs>' 'lib.systems.examples.remarkable1'
  1. Create a non-root user on the tablet, e.g. useradd siraben && passwd siraben. Ensure that you have passwordless SSH set up by using ssh-copy-id.
  2. The root partition on the tablet has very limited space (22 MB), so, as root, mkdir -p /nix /opt/nix && mount --bind /opt/nix /nix. The bind can be made persistent by adding the following line to /etc/fstab
/opt/nix /nix none bind,nofail 0,0
  1. Install Nix on the device. I used GitHub - DavHau/nix-on-armv7l: Nix without NixOS on arm32v7 / armv7l / Raspberry Pi OS / Raspbian and decompressed the tarball in the releases and ran the install script.
  2. Using nix-build and nix-copy-closure, one can cross-build from their machine and transfer it to the tablet, like so. The NIX_SSHOPTS is needed because nix isn’t available unless .profile is sourced.
$ NIX_SSHOPTS="source .profile;" nix-copy-closure --to siraben@10.11.99.1 "$(nix-build -f '<nixpkgs>' -A pkgs.pkgsCross.remarkable1.hello)"

Happy hacking!

4 Likes

So glad to find someone doing this. I’ve had my eye on the Remarkable for a while, but they never answer my questions on whether the system is open to outside development and what tools will be available. I think I’ll buy one now!

1 Like

Also see GitHub - reHackable/awesome-reMarkable: A curated list of projects related to the reMarkable tablet. Worth noting reMarkable company provides toolchain and is responsive on their linux repo, but cannot provide support for third party mods.

1 Like

Based on this work, reMarkable 2 cross-compilation has also been added. Cross-compile configuration for reMarkable 2 tablet by tadfisher · Pull Request #105432 · NixOS/nixpkgs · GitHub

2 Likes

Just wanted to say thanks, got my reMarkable 2 today and immediately got Nix to work on it. Feels like living in the future :slight_smile:

4 Likes

@Manveru curious how you did it: I’ve got a remarkble 2 with this week update to 2.26 . Install of nix fails because of invalid option -- 'c'

reMarkable: ~/nix-2.3.7pre0_0000000-armv7l-linux/ ./install-multi-user   
...
We will:

 - make sure your computer doesn't already have Nix files
   (if it does, I will tell you how to clean them up.)
 - create local users (see the list above for the users we'll make)
 - create a local group (nixbld)
 - install Nix in to /nix
 - create a configuration file in /etc/nix
 - set up the "default profile" by creating some Nix-related files in
   /home/root
 - load and start a service (at /etc/systemd/system/nix-daemon.service
   and /etc/systemd/system/nix-daemon.socket) for nix-daemon

Ready to continue?
[y/n] y

head: invalid option -- 'c'
BusyBox v1.31.1 (2020-12-17 19:42:00 UTC) multi-call binary.

Usage: head [OPTIONS] [FILE]...
head: invalid option -- 'c'
BusyBox v1.31.1 (2020-12-17 19:42:00 UTC) multi-call binary.

Usage: head [OPTIONS] [FILE]...

It seems that the install script is not POSIX compliant: there’s no -c option in POSIX head (c.f. head).
There are only a couple of occurences of head -c in the script, and mainly for display purposes. I believe you can replace these with head -n 1 and get roughly the same experience.

However, there’s another issue that I am confused by.

@siraben, what is the point of creating another non-root user?

Also, with which user do you install nix with? root?
It seems that sudo is not installed on the rM2 out of the box…

My guess is that you installed with root, and did a multi-user install. Am I correct?