Upcoming Hydra Schema Change

Announcements
1

Hello everyone,

Just dropping a note for people who have deployed their own Hydra. This is advanced notice on an upcoming migration to Hydra’s schema. Almost all migrations to Hydra’s database are fast and performed automatically by hydra-init when starting the server. However, this migration will potentially take many hours and possibly over a day for large deployments. Because of that, we have designed the migration as a two-phased deployment, with a command-line program which will run the slowest part of the migration while the server is running.

This means, however, that Hydra admins must take care in the deployment and will ideally do the deploy in two phases.

Some information about the problem and what we’re doing: Give Jobsets an ID; add jobset_id to builds and jobs. by grahamc · Pull Request #710 · NixOS/hydra · GitHub if you have any feedback on how this is being done or ways to make it easier on Hydra admins, please do comment.

Thanks,
Graham

6 Likes
2

Hey, thanks for your work and the advanced notice.

Does every hydra installation need to go through the two-phased deployment, even if it is relatively small or can live with the downtime?

Edit: I ended up executing your queries by hand, after all.

3

@grahamc: I have a few questions about how this migration impacts server downtime. Specifically:

  1. Is the Hydra server functional after selecting the pkgs.hydra-migration package and before running the hydra-backfill-ids script?

  2. Is the Hydra server functional while running the hydra-backfill-ids script?

  3. Is the Hydra server functional after running the hydra-backfill-ids script but before selecting the pkgs.hydra-unstable package?

3 Likes
4

@grahamc answered my question here, which I’ll copy to this thread:

the migration is completely online, except for the brief period of time where the columns are added (a few seconds in the first migration) and indices are added (up to a few minutes, second migration.)

2 Likes