I see the latest AMI (Amazon Machine Image) is for the NixOS-23.05 release.
Is there anything holding up a release for a NixOS-23.11 AMI?
2 Likes
Theoretically speaking, pinging @AmineChikhaoui is part of the release process for 23.11 to start the process of creating new AMIs: Branch-off - NixOS Release Wiki.
But given the state of maintenance of our AMIs (alas), I am not very comfortable continuing this process, hence why I opened feat: remove AMI uploading of the process by RaitoBezarius · Pull Request #75 · NixOS/release-wiki · GitHub and nixos/amazon: drop by RaitoBezarius · Pull Request #269240 · NixOS/nixpkgs · GitHub.
@arianvp proposed himself (with the help of @tomberek) to step up and take over this work (as can be seen in those PRs).
That’s all the information I have, at the moment, on the matter. I will leave it to the people working on that to update us on the status.
cc @figsoda
2 Likes
I’m slowly working on fully automating the AMI upload process here GitHub - NixOS/amis: Temporary home for the soon to be official NixOS AMIs (super WIP). Aiming to have it ready for 24.05 but hopefully earlier. Different than before I’ll be uploading AMIs for every channel bump. With it automated it should remove the burden on release managers to worry about uploads and AWS credentials.
I’m also in contact with the infrastructure team to get my own AWS sub-account under NixOS’ AWS Org to start uploading these and they will then be the new official AMIs.
If @AmineChikhaoui doesn’t have time I’m happy to help upload the 23.11 AMIs through the old processs.
4 Likes
I actually don’t mind uploading the AMIs, it’s just that afaik nobody pinged me in this release.
Btw I didn’t follow the discussion much, is the issues with AMIs in the build process ? as the upload of AMIs is generally easy and I even attempted automating that in rewrite nixpkgs's create-amis script using Terraform by AmineChikhaoui · Pull Request #254 · NixOS/infra · GitHub .
2 Likes
Little update on this
The NixOS infra team created a new AWS account for AMIs (as we were uncomfortable giving me access to the account they’re currently uploaded in as that hosts also other stuff that I don’t want access to at all).
They have given me access to that. And I am using it as a testing ground now to get automation in place to upload AMIs for each channel bump automatically; including a browser web page where you can find all the AMIs. However I am planning for that work to be finished for 24.05.
I can give a shot at uploading the current 23.11 AMIs to the new account when I’m back from Holidays and then update the AMI ids in nixpkgs. However @AmineChikhaoui if you have time to do this and still have access to the original account it’d be one less thing to worry about. Sorry to hear you didn’t get pinged during the release process. I wasn’t aware of that.
By the way @AmineChikhaoui I did see your work on the terraform automation. I took at least inspiration of using github actions + OIDC to make things easy to automate. But I decided to use the AWS SDK directly instead of Terraform in the hope that I can switch to the EBS-Direct API for uploading the snapshots later. Importing the snapshot and copying them across accounts is the bottleneck and they can be made a lot faster with the EBS-direct API and Terraform doesn’t (and probably won’t) expose it. However it’s all future work of course. Step 1 is to get it to work now.
5 Likes
Thanks Arian!
BTW I submitted create-nixos-amis: packages nixos/maintainers/scripts/ec2/create-amis.sh by basvandijk · Pull Request #277079 · NixOS/nixpkgs · GitHub since I also needed a way to create custom NixOS-based AMIs. Not sure if that conflicts with your work.
Now the entrances to create AWS Ec2 is not available in Download Nix / NixOS | Nix & NixOS (the region drop down list is empty).
I think we’d better just rollback to NixOS 23.05 for now and update it once the 23.11 AMI is ready.
1 Like
I have a PR open to add the 23.11 AMIs. This will also fix the website.
There’s a WIP website for searching the AMIs here: NixOS Amazon Images / AMIs
These will get updated for every channel bump.
We’re working on integrating this page to the main nixos website
3 Likes