Thanks, I may have read that somewhere, but 1.6.1 was indeed a long time ago.
Do you think this approach would be viable at the moment?
I mean, not sure exactly what the minimum configuration is for a NixOS machine to become a deployment target other than
(Found also this issue but the gentleperson there was using NixOps from a Ubuntu machine that may have had to do with the troubles.)
Sorry for the laggy relies, I’m trying to integrate Discourse into my routine…
Regarding Gen2 VMs: Why Choose Gen2? aka Timeline for Gen2 Disk Encryption? · Issue #52340 · MicrosoftDocs/azure-docs · GitHub. By their own admission, Gen2 VMs support “modern booting via UEFI” (that has zero net benefit afaict) and lack support for Azure Disk Encryption. Normally I like to aim for where the ball is going to be, but I’m not going to support launching features like this, so I don’t intend to look at Gen2/UEFI support until ADE is supported.
No. Unfortunately all of that crap in azure-new now is the bare minimum to get a Managed Disk uploaded for a single subscription. There is no way to publish a Managed Disk publicly and I don’t want to re-write legacy code to deal with publishing a public disk blob in an Azure Storage Account (and the code to create/ensure the account and container, and then re-write more code to import that disk to the end-users subscription for it to actually be usable.) Frankly, I’m tired of talking and typing about it and working around it. I wish they would just pay someone to actually use their APIs and give them internal feedback about this kind of stuff.
After using and tinkering with
azure-newI kind of figured it out that it is more of a template than a tool, but a very handy one for newcomers like me. The most important part of it is the VM creation itself that would’ve taken me a long time to figure out (and not I wouldn’t even dare touch it at the moment), and the shell scripts are going to be modified by anyone based on their taste anyway. Thanks for figuring out all the steps and quirks, and documenting them.
Let me know if you think any quirks are worth incorporating. I’m happy to iterate on azure-new, review PRs, whatever. I’m glad to hear they’re of use.
For now, I am using azure-new scripts along with image definitions (that have sudo changes, my own services, etc) in my own nixcfg repo to build images and deploy instances. I update the VMs either by making it a devbox and cloning my nixcfg+nixpkgs, or I have my own “remote deploy” script that builds a machine closure, copies and activates it (similar to what nixops would do).
Not sure about others.
My limited understanding of NixOps Azure is:
I suspect significant portions would need revamping or rewriting. I’ve heard rumors of someone having interest in reviving a nixops-azure, but nothing concrete.
but now it seems that there are fundamental issues with Azure that I have yet to find out about.
. I’ll admit, I’m being somewhat stubborn. But it is considerably easier to upload images to GCP/AWS and considerably easier to share them publicly afterward. Less API calls, less concepts to learn, less waiting on slow APIs, less spending an hour figuring out how to name storage accounts, on and on.
But no, there’s nothing that fundamentally prevents a good NixOps + Azure experience if someone were to write the nixops backend. (Ping me if you are interested…)
I think in theory this could/should work? I am actually looking to adopt NixOps to manage my machines (which would include my managed Azure machine) and in theory I already manage them all the same way, so it ought to work.
Trust me when I say that everyone is aware of the pain when they onboard onto a team, but customer usability is usually secondary to feature development.
Not to mention that teams usually own a very small slice of the big public-cloud pie, so they develop on top of the work of other established teams (e.g. blob storage), or they roll their own (usually overlapping) solutions for a given user scenario (e.g. images).
That’s my impression as well.
Thank you so much for the comprehensive answers! … and I’m sorry for the late reply as well.
I am meaning to do a deep-dive in your nixcfg, and thanks for describing your workflow. Now I have concrete ideas where to start, and this is indeed almost identical to “deploy with azure-new, manage with NixOps”; especially nixup.sh looking very similar in functionality to what NixOps does. (I was meaning to ask you about this so thanks for pre-emptive answer.)
Two weeks ago, I spent almost an entire day following all the threads, and this effort seems to be actively dead. (Pardon the oxymoron.)
I guess I’m quite “lucky” then that I don’t have experience with custom VMs on GCP/AWS - if we ever going to switch providers, it will just get easier.
Would love to, but there’s only a slim chance of ever having such abundance of free time. (But then I already put it on the list, so thanks again.)
Just an update on this: made a fork of the azure-new script that we are using internally but @colemickens made another (and more modern) one at GitHub - colemickens/flake-azure-demo at dev (not sure about its status as I am still at the level of struggling to understand flakes).
There is also a #nixos-azure IRC channel with logs.)
My MSDN subscription has finally reached some sort of hidden expiration date (far later than it probably should have). This subscription had been providing me with $150/month in Azure credits, which was enough to motivate me to keep some long-running services in Azure, and thus author nixos-azure (check the dev branch, the readme is stale, but the nix code is all there), a smaller, better Rust Azure boot agent, and write scripts for reliable image publishing.
But, now that that free subscription is gone, I have no reason to use or support Azure. (If this changes again, I’ll reply again here.)
@colemickens your efforts have definitely improved the situation. I was a little saddened by the ease of using pre-built images (or lack thereof) when I was working at Microsoft.
hello everybody,
just giving this a push… did anybody investigate NixOS on Azure Gen 2 VMs?
Given the lack of replies, my guess is no, but I’m planning to (i.e., have to) get back to it in a month or two the latest so thanks for the bump!
we actually made it work in the meantime.
I hope we find the time to open source the missing pieces. Ping me if I forget about it 
Here’s a minimal working example:
Gonna upstream this into nixpkgs, at some point when I have a bit of time over.
Tangentially related: Started re-reading Azure’s Virtual Machines Documentation, and this (well-buried) part is a deal breaker as I wanted to deploy it for a job:
The Azure platform SLA applies to virtual machines running the Linux OS only when one of the endorsed distributions is used.
I want to run a nixos machine on a azure vm. What’s the current status? Is plommonsorbets minimal example the way to go or has this been upstreamed into nixpkgs?
As far as I understand by now, it works like this:
Azure supports two kinds of VM images: v1 and v2. The main difference is that v1 use BIOS and v2 boot with UEFI. v2 has some more security features and neither v1 nor v2 are supported by all instance types Azure V1 vs V2.
Support to generate images for v1 is in nixpkgs / nixos here. The example from plommonsorbets shows how to create a v2 image.
Once you have an image there is the question on how to get it into azure. Which seems kind of journey in itself (most of Azures documentation wants to stuff Azure Image Gallery down your throat). But the azure-new script shows you how to get from a vhd to an image.
There was (is?) an attempt at making azure-new more user friendly:
(The background stuff in the readme can be ignored.)
It’s basically azure-new extracted from Nixpkgs to allow personal experimentation without being embedded in a huge repo. The section 2. Usage gives a good overview, and there are more comments in the shell scripts.
Took me a while, because uploading the image always timed out due to my rather slow internet connection at home, but
[azureuser@localhost:~]$ cat /etc/issue
<<< Welcome to NixOS 23.11.20230801.9e1960b (\m) - \l >>>
The stock v1 images in nixpkgs work like a charm
Hi, am trying to run nixos in Azure and using the v1 image build of azure-new. It successfully runs after deploy but as soon as i do a nixos-rebuild and reboot the VM i get the error: 452: Out of range pointer: 0x100010
am i not supposed to run a nixos-rebuild or what am i missing here?
So i did some testing and finally got a working image which i can rebuild and reboot without any issues on gen 2.
I combined the 2 repositories azure-new and plommonsorbets with some extra config on cloud-init and networking.
If anyone is interested :
Works like a charm. Thanks for this. ( you are my hero for today.)
I did this recently fairly trivially using nixos generators. I generated the vhd using the default settings, then:
az storage account create