Podman logs erroring with default journald logDriver

I’m using these settings to setup Podman on my computer:

{ config, pkgs, ... }:
{
  virtualisation = {
    podman = {
      enable = true;
      dockerCompat = true;
      dockerSocket.enable = true;
      extraPackages = [ pkgs.zfs ];
      defaultNetwork.dnsname.enable = true;
    };

    # Rootless podman on ZFS
    containers.storage.settings = {
      storage = {
        driver = "zfs";
        graphroot = "/var/lib/containers/storage";
        runroot = "/run/containers/storage";
      };
    };

  # Disable NixOs Containers (conflicts with virtualisation.containers)
  boot.enableContainers = false;

  # Disable firewall, so other machines can access the containers
  networking.firewall.enable = false;

  # Handy packages
  environment.systemPackages = with pkgs; [
    podman
    podman-compose
    podman-tui
  ];
}

And I can run containers, but when I try to get the logs I get an error:

$ podman run hello-world:latest
# omitting the output here

$ podman logs vigorous_ellis
Error: initial journal cursor: failed to get cursor: cannot assign requested address

I tried to google this error, but couldn’t find what is the cause. I also tried to change the log driver to k8s-file to no success, with:

    virtualisation.containers.containersConf.settings = {
      log_driver = "k8s-file";
    };

How can I troubleshoot and fix this?

That’s weird, but after adding my user to the systemd-journal group I can access the logs.

I haven’t reached down exactly why but on one server I run a rootless podman setup I just started getting this as well.

I updated from 22.05 to 22.11 and made a few config changes so I have to go back and isolate what caused this still…

Any other information you can provide that might help me?

Thanks!

Check if your user is in the systemd-journal groups It should give access to the journal logs.

According to the journalctl man page, wheel group should also give permissions to read the logs, but in my case it did not work…

I still don’t know why.