Nextcloud Memories app exiftool errors

firecat53 · October 25, 2023, 4:57pm

Is there any running Nextcloud with the Memories app that is seeing this bug? It seems likely to be Nix related but doesn’t seem to affect the actual indexing by the memories app.

Thanks!

Shawn8901 · October 25, 2023, 6:22pm

Hi, I am not using the app (but nextcloud). According to the logs on the GitHub issue you possibly want to add perl to the path of the nextcloud phpfm unit (or what ever is running the app code) According the config screenshot it should use the systems perl for some operation, but that is likely not on the path, which leads to the perl not found log. here is an example how you can extend the path of a systems unit. (Just random code from my config and that was the easiest to find for me atm)
You can also check if the logs go away if you uncheck the option about the perl not found (as you said everything is working and it seems to be a fallback anyways according to the wording)

firecat53 · October 25, 2023, 10:08pm

  systemd.services.phpfpm-nextcloud = {
    serviceConfig = {
      PrivateDevices = lib.mkForce false;
    };
    path = [pkgs.perl];
  };

Tried that, but it had no effect. When I uncheck the ‘use system perl’ option, it shows the error on the config screen exiftool failed test: failed to run exiftool: /tmp/exiftool-ocgbk6len1yp-12.60-239ca57e -ver. and the following logs:

Oct 25 15:04:25 homeserver Nextcloud[235541]: {"reqId":"EyjLhSC60RGGDVwxNBFa","level":3,"time":"2023-10-25T22:04:25+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"proc_open(): Exec failed: No such file or directory at /var/lib/nextcloud/store-apps/memories/lib/Exif.php#385","userAgent":"--","version":"27.1.2.1","data":{"app":"PHP"}}
Oct 25 15:04:25 homeserver php[235540]: WARN: Failed to create stay_open exiftool process
Oct 25 15:04:25 homeserver php[235542]: sh: line 1: /tmp/exiftool-ocgbk6len1yp-12.60-239ca57e: cannot execute: required file not found
Oct 25 15:04:25 homeserver Nextcloud[235540]: {"reqId":"BsOnmkui17VgxrqgPKlp","level":3,"time":"2023-10-25T22:04:25+00:00","remoteAddr":"","user":"--","app":"memories","method":"","url":"--","message":"Memories: Indexing exception: failed to run exiftool: /tmp/exiftool-ocgbk6len1yp-12.60-239ca57e -ver","userAgent":"--","version":"27.1.2.1","data":{"app":"memories"}}

I didn’t try adding the path=[pkgs.perl] directive to any other services because I’m almost positive the php processes are run by phpfpm-nextcloud.service.

Thanks for the thoughts! Any other ideas?

Shawn8901 · October 26, 2023, 8:26am

I have currently no access to my machines, I will be back at the end of the weekend, and take a look when I am back at home if no one else has taken a look.

Shawn8901 · October 30, 2023, 8:15pm

So i looked at it.
Basically in the upstream issue there was also the kind of the solution posted.
The perl needs to be in the nextcloud-cron.

I did basically add

  services = {
    nextcloud.extraOptions."memories.exiftool" = "${lib.getExe pkgs.exiftool}";
    nextcloud.extraOptions."memories.vod.ffmpeg" = "${lib.getExe pkgs.ffmpeg-headless}";
    nextcloud.extraOptions."memories.vod.ffprobe" = "${pkgs.ffmpeg-headless}/bin/ffprobe";
  };
  systemd.services.nextcloud-cron = {
    path = [pkgs.perl];
  };

in addition to my other changes and got it showing green after waiting for a next cron run

 Automatic Indexing status: Indexing completed successfully

firecat53 · October 31, 2023, 3:50pm

Thanks so much for taking a look at this! A couple of issues still after I start playing around some more:

Setting the paths via nextcloud extraOptions puts them into an array like this:

'memories' =>
  array (
    'exiftool' => '/nix/store/4zgj1wjp8bnrzl2ridf9y1ib6214w77p-perl5.38.0-Image-ExifTool-12.65/bin/exiftool',
    'vod' =>
    array (
      'ffmpeg' => '/nix/store/3aad4882pqrc05ssnz08dipa3wmbckl7-ffmpeg-headless-5.1.3-bin/bin/ffmpeg',
      'ffprobe' => '/nix/store/k4j0i7w6h59nmd8pwfd0lwgyj3s9nb0n-ffmpeg-5.1.3-bin/bin/ffprobe',
    ),
  ),
  'memories.exiftool_no_local' => true,
  'memories.vod.path' => '/var/lib/nextcloud/store-apps/memories/bin-ext/go-vod-amd64',

Nextcloud doesn’t seem to like that. It won’t find and fill the correct paths in on the admin page until I add those values outside of the array like this:

  'memories.exiftool' => '/nix/store/4zgj1wjp8bnrzl2ridf9y1ib6214w77p-perl5.38.0-Image-ExifTool-12.65/bin/exiftool',
  'memories.vod.ffmpeg' => '/nix/store/3aad4882pqrc05ssnz08dipa3wmbckl7-ffmpeg-headless-5.1.3-bin/bin/ffmpeg',
  'memories.vod.ffprobe' => '/nix/store/k4j0i7w6h59nmd8pwfd0lwgyj3s9nb0n-ffmpeg-5.1.3-bin/bin/ffprobe',
  'memories.exiftool_no_local' => true,
  'memories.vod.path' => '/var/lib/nextcloud/store-apps/memories/bin-ext/go-vod-amd64',

Is the video thumbnailing working for you? I have the “ffmpeg preview binary not found” error on the admin page, and it seems like video thumbnails are not working.

EDIT: fixed this by adding preview_ffmpeg_path = "${pkgs.ffmpeg-headless}/bin/ffmpeg"; to extraOptions. Couldn’t use lib.getExe because it returns ffmpeg-headless when recognize specifically looks for ffmpeg.

For the “Recognize” app, the only way I can get it to work is to install nodejs_18 systemwide and add the path on the admin page /run/current-system/sw/bin/node. There don’t seem to be any config.php options for Recognize to set paths like there are for Memories. Any thoughts on that?

Thanks!

Carrot · October 31, 2023, 6:45pm

I’m planning to use nextcloud soon. I’ve looked at this config for inspiration:

github.com

rafaelsgirao/nixos-config/blob/1dc5171e946b82ba833f223cdb5a8e9491093173/modules/nextcloud.nix

{ config, pkgs, lib, hostSecretsDir, ... }:
let
  # fqdn = config.networking.fqdn;
  ncHost = "cloud.rafael.ovh";
  altHost = "cloud.spy.rafael.ovh";
in
{

  age.secrets = {
    NC-adminpass = {
      file = "${hostSecretsDir}/Nextcloud-adminpass.age";
      owner = "nextcloud";
    };
    # NC-secretfile = {
    #   file = "${hostSecretsDir}/Nextcloud-secretfile.age";
    #   owner = "nextcloud";
    # };
  };
  #Nextcloud
  services.nextcloud = {

This file has been truncated. show original

Shawn8901 · October 31, 2023, 11:01pm

firecat53:

Thanks so much for taking a look at this! A couple of issues still after I start playing around some more:

Setting the paths via nextcloud extraOptions puts them into an array like this:

'memories' =>
  array (
    'exiftool' => '/nix/store/4zgj1wjp8bnrzl2ridf9y1ib6214w77p-perl5.38.0-Image-ExifTool-12.65/bin/exiftool',
    'vod' =>
    array (
      'ffmpeg' => '/nix/store/3aad4882pqrc05ssnz08dipa3wmbckl7-ffmpeg-headless-5.1.3-bin/bin/ffmpeg',
      'ffprobe' => '/nix/store/k4j0i7w6h59nmd8pwfd0lwgyj3s9nb0n-ffmpeg-5.1.3-bin/bin/ffprobe',
    ),
  ),
  'memories.exiftool_no_local' => true,
  'memories.vod.path' => '/var/lib/nextcloud/store-apps/memories/bin-ext/go-vod-amd64',

Nextcloud doesn’t seem to like that. It won’t find and fill the correct paths in on the admin page until I add those values outside of the array like this:

  'memories.exiftool' => '/nix/store/4zgj1wjp8bnrzl2ridf9y1ib6214w77p-perl5.38.0-Image-ExifTool-12.65/bin/exiftool',
  'memories.vod.ffmpeg' => '/nix/store/3aad4882pqrc05ssnz08dipa3wmbckl7-ffmpeg-headless-5.1.3-bin/bin/ffmpeg',
  'memories.vod.ffprobe' => '/nix/store/k4j0i7w6h59nmd8pwfd0lwgyj3s9nb0n-ffmpeg-5.1.3-bin/bin/ffprobe',
  'memories.exiftool_no_local' => true,
  'memories.vod.path' => '/var/lib/nextcloud/store-apps/memories/bin-ext/go-vod-amd64',

For me (nixos-unstable) it works fine when declaring it like this

    nextcloud.extraOptions."memories.vod.ffmpeg" = "${lib.getExe pkgs.ffmpeg-headless}";
    nextcloud.extraOptions."memories.vod.ffprobe" = "${pkgs.ffmpeg-headless}/bin/ffprobe";

  'memories.vod.path' => '/nix/store/yykmsgb1w3yl59l8yp4131qgnl5xbrh9-nextcloud-app-memories-5.5.1/exiftool-bin/go-vod-amd64',
  'memories.exiftool' => '/nix/store/xzcpvirivxfkqmnbxcrar9lqbby3m64g-perl5.38.0-Image-ExifTool-12.68/bin/exiftool',
  'memories.vod.ffmpeg' => '/nix/store/fxg8v7zfq6fan7fnmgsixlpizalqdrsw-ffmpeg-headless-6.0-bin/bin/ffmpeg',
  'memories.vod.ffprobe' => '/nix/store/fxg8v7zfq6fan7fnmgsixlpizalqdrsw-ffmpeg-headless-6.0-bin/bin/ffprobe',
  'memories.exiftool_no_local' => true,

Are you by chance using nixos stable? The mainProgram was set just set last month, so that could be the reason that lib.getExe is not working for you

I did the same so far. According the source code its looking for a app config, and those are stored in the database according nextclouds developer docs.
So select * from oc_appconfig; is showing them, thus setting it to /run/current-system/sw/bin/node seems to be the best solution.

So all the status things are green aside libtensorflow for me atm. If its really fully working i can currently not say, as i was not using nextcloud for storing photos, but ill upload some tomorrow and will report back.

firecat53 · November 1, 2023, 4:51am

@Shawn8901 Thanks for the feedback! Yes, I am using stable. Everything is setup and seems to be working ok now. I’ll be interested to see if the paths get updated correctly after the next ffmpeg update.

@Carrot Thanks! I’m stealing a couple of ideas, most notably the DeviceAllow and adding the preview providers to extraOptions.

anticipatedfart · January 24, 2024, 8:17pm

I have the same problem so I started a new topic.

Toomoch · July 6, 2024, 9:19pm

Hi, did you get the DeviceAllow option working with VAAPI? It only works for me if I set PrivateDevices = lib.mkForce false.

firecat53 · July 6, 2024, 11:59pm

No, it doesn’t seem to work with the DeviceAllow statement, just with the PrivateDevices statement. I tried it once (just now, cause I forgot!) and it didn’t work, so reverted that change. I didn’t do any other research or troubleshooting.