Asahi Nixos on external drive

Hi,

I am once again experimenting with asahi linux and Nixos on a Mac M1.
Storage is a concern on macbooks as it is not replaceable, and I am facing issues with my dual boot MacOS with nix / Asahi with nixos. My 500gb internal drive is always full as nix store takes a lot of space and I need to share theses 500gb with two systems.

So I got the idea (and I must not be the first) to move asahi to an external drive. After some quick search, it seems that asahi is not really ready to be moved fully to an external drive. Macbooks does not support external boot partitions, and it seems there is issues when the boot partition is on the internal drive and the root on an external drive (something with usb drivers when leaving sleep mode).

So my questions are:

  • Would it be possible to move the nix store to the external device while keeping a small sized root partition on the internal drive ? (IE: Are everything needed by the system to leave sleep mode located on the root partition, or do some important things resides in the store ?)
  • Even better, would it be possible to share the same store partition for both systems ? (I know packages built for linux-arm wonā€™t be usable on darwin-arm but it would save me the hasle to have two separate partitions that would each eat some space on the disk even when not full)

Thankā€™s for any help,

JM

Even better, would it be possible to share the same store partition for both systems ?

as you observed, the software for linux and darwin is not the same, so you would have one store with twice as many store paths, saving no space. Besides, sharing a store breaks the gc: a gc root for a system is not valid for the other one, so running the gc will uninstall the other system, so to speak

1 Like

More specifically the gc roots wonā€™t point at valid symlinks since the mounts are probably different. If you wanted to do this sort of thing, youā€™d have write your own app for gc.

Having actually tinkered with a mac M1 for a bit its tricky.

The partitioning for those M1 macs is completely different. Apple must have been smoking some really good stuff when they wrote it.
The partition on the apple fabric needs to be cryptographically signed by an onboard chip inside the mac, that chip ships with boot keys.

You probably noticed this partition, dont mess with it, dont resize it, and dont delete it or you will perma brick your machine.

The only solution to fixing that specific problem is to wire it into another Mac, an M1 specifically to restore it. So if you dont have another mac to undo this potential problem, dont mess with your boot keys and try not to mess with the Fabric.

Apples firmware also does some extra checks on boot to validate the fabric and some of the data on it, including the mac bar. Context: Way back when HDD space was MBā€™s total, and saving a kb was a big deal, apple moved some functions of the Macintosh OS into a chip soldered onto the board, some of this includes boot keys and the apple bar. If you break it, that mac will NEVER boot again.
If you have are successful with a dual boot, a cleaner work around specifically for an M1 mac is to make a new resizable disk image thats mountable in macOS and writable in asashi, then you can mount this disk in Ashahi linux as a partition. This effectively shares the store between the 2, then you can ā€˜tryā€™ installing nix darwin to that partition stack and attempt to set it as root for nix in asashi but i will warn you, this is likely to have someā€¦ unintended consequences.

2 Likes

Ok, I will forget about that ideaā€¦

Yes, I wonā€™t try to fully boot from external drive, Iā€™m not confident enough in my understanding of the mac boot processā€¦

I will give a try to the external nix store. Maybe first with a non-nixos system with nix as a package manager.

Hereā€™s some detailed info from apple about the M1 boot process.

Youā€™ll also want to thoroughly read this.
TLDR disable SIP if you wanna tinker much.

And for if you make an oopsy, here are your bailout instructions for restoring the firmware of an M1

If youā€™re going to try a non Nix system, asahi Linux is really the only one thatā€™s got decent support for the hardware. I wouldnā€™t stray too much from there or you are firmly on your own

Hi,

Iā€™m back after some trials and errors.
I decided to forget about asahi for now and first have a working setup for nix on MacOS with external store.

Found that thread and was able to install nix and its store on the external drive. However Iā€™m facing issues when unplugging my drive (to transport the computer for example).

When the external drive is unplugged then plugged back, all nix commands are failing with a ā€œcannot connect to socketā€ error and I need to reboot to fix the issue.

I feel like its a nix daemon related issue, as the socket is in the /nix partition, maybe the daemon canā€™t find it after a disk unplug. Is there a way to manually restart the daemon without restarting my entire system ?