Video upscaling with a Nix derivation

Help
1

Hello. Some day ago, I wanted to use an animated wallpaper for plasma (replacing the previous one). However, this was an animated GIF image that is at a lower resolution than the one of my screen (and with very visible compression artifact) (if anyone is interest… #2606077 - safe, artist:smowu, oc, oc:lacy, oc:seeker, bat pony, dog, dog pony, pegasus, pony, animated, autumn, bridge, car, cinemagraph, crepuscular rays, detailed background, looped, mouth hold, radiation sign, rope, scenery, scenery porn, tail wag, technically advanced, tree, truck, tug of war, tugging, wings - Derpibooru ). I anyway added it to my my wallpaper folder with home-manager and fetchurl.

I then looked at tool that could help enhance the quality of the image. I started using Anime4K (not managed via Nix) a few month ago, and have been really happy for upscaling video at resolution lower or equal to 720p (my screen is at 1080p). After taking a look, I found waifu2x to be the best for this kind of stuff (and the only upscaling software packaged in Nix).

As I like to play with Nix, I started making a nix derivation that would upscale the input video, and make it loop. I based part of my work on my previous experience using Nix for computing another animated wallpaper I hacked with python and inkscape ( https://github.com/marius851000/MLP-loyalty-animated/blob/a1f5d10f1d0ebbe9e7d1315b0a840dd2498c7fd7/default.nix ).

Waifu2x only apply on each frame idividually (don’t directly work on video) so I followed those step (derivation)

  • get video metadata (number of frame and framerate) (depend on input video)
  • extract each frame as a PNG image (single derivation) (depend on input video)
  • for each frame (depend on the metadata), upscale it with waifu2x (depend on extracted frame). There is one derivation per frame (making use of… not sure the name, but dynamic resolution or something like that).
  • combine all the frame in a final video at the correct frame rate (depend on every frames and the metadata)

This take some input parameter. Noteworthy is additional_waifu_args = "-m noise-scale --noise-level 3";, usefull for badly compressed GIF (still cause some artifact). and loop, that loop the video, and default to true.

The code is avalaible here lib/upscale_video.nix · master · marius david / mariusnur · GitLab and is avalaible in my nur ( marius david / mariusnur · GitLab ) with the path lib.upscale_video. Additionally, I have a nix flake that can be used to use animated image wallpaper with plasma ( GitHub - marius851000/ricenur: a Nix User Repository containing various stuff for Nixos ricing ).

Nice aspect of Nix (in addition to theorical reproducibility) is that “automatic” parallelism (well, still needed to package it) and possibility of sharing the work between multiple computer.

I have some question concerning Nix for this kind of work, thought :

  • Is it possible to make platform-indendant derivation ? This result should in theory be independant of the hardware used to compute it, but I see many problem that can arise with nix if trying to use multiple CPU family interchangeably. This can be usefull for distributing load with other computer CPU (I have my rasberry pi at home, thought it’s likely not powerfull enought to make an important difference. But with a cluster with Apple, FreeBDS and Linux computer, this could lead to some gain).
    (badly wored, see this comment Video upscaling with a Nix derivation - #3 by marius851000 )
  • Does Nix cache evaluation at a lower level than flake level and derivation level ?
  • Also, do you have any general tips for derivation that output data/images ?
1 Like
2

Can’t comment on the animation/video/image editing side of things, but as for your questions:

Is it possible to make platform-indendant derivation ?

Chapter 9. Cross-compilation of the Nixpkgs manual covers this topic in detail. I think the gist is that the majority of Nix expressions will result in a platform-independent derivation, and the compilers during the build process will get called with architecture-specific flags.

Hydra (the Nix continuous build system) automates this with “the system parameter refers to the system architecture that we want to build for, which can be (for example) 32-bit Linux machines, 64-bit Linux machines, 64-bit Mac OS X and so on.”. You check the end results yourself on the public Hydra instances (that I know of): Nix’s hydra.nixos.org (e.g., see job name here) and flox’s Storehouse (see the naming of the list of jobs again).

Also, do you have any general tips for derivation that output data/images ?

Can only say that this can definitely be done; at least seeing all the stuff people have come up with regarding Nix. (By the way, thank you for linking your code!)

Does Nix cache evaluation at a lower level than flake level and derivation level ?

Someone more knowledgeable has to chime in regarding this…

3

Thank ! For the first question, I have probably badly exposed it (sorry). I’m aware it is possible to write package definition in a cross-compilation friendly way (and sometimes check that packages I made can be cross compiled). What I wanted to ask is: Is there a way that a derivation build using x86-64-linux waifu2x (or any other platform agnostic data producing tool), when served by a binary cache, be downloaded by a aarch64-darwin and reuse it.

The main obstacle to this, AFAIK, is the depency hashing. waifu2x isn’t the same hash (as it is a different platform), and so, won’t fetch it, as the result hash would also be different (if using input addressed hash. The final hash would be the same if using CA derivation, but it can’t know it until it build at, at which point downloading it is useless). I think this can be worked around by hardcoding a hash to the derivation, but this doesn’t look a good idea when all inputs are supposed to be deterministic.

1 Like
5

Yes. There isn’t a way because that’s already how it works.

The thing you need to do is to get a derivation with a x86_64-linux system attribute on an aarch64-darwin machine. To do that, pass system = "x86_64-linux"; when importing nixpkgs (same version on both systems) and your derivation should evaluate to the same thing as one evaluated on a x86_64-linux machine. You can’t build this derivation (because it has the wrong system), but if it’s available in binary cache Nix would happily use it.