Very high fan noises on nixOS using a system76 thelio

spacebuffer · December 9, 2022, 3:41pm

I have a thelio, popOS was on it before and everything was perfect. when I switched to nixos I noticed the fans were always running at a fast speed even when I am not doing anything.

so I enabled this option hardware.system76.enable.all = true and rebooted, a few good things happened (that I noticed)

the packages system76-power and system76-firmware were installed
their services were enabled respectively

the bad things I noticed:

the fans were still loud
running system76-power profile returned ? where on popOS it returned balanced if I remember correctly

I tried running system76-power daemon which is the command that is supposed to be running as a service after enabling the enable.all option before, and a few things happened once again:

the fans suddenly went quite, at the time I wasn’t sure if they turned off completly or if they just went down temporarly because the temps were low
I got a bunch of errors

[nix-shell:~]$ sudo system76-power daemon
[INFO] Starting daemon
[INFO] Connecting to dbus system bus
[INFO] Rescanning PCI bus
[INFO] 0000:09:00.0: AMD graphics
[INFO] 0000:09:00.0: Function for 0000:09:00.0
[INFO] 0000:09:00.1: Function for 0000:09:00.0
[INFO] Disabling NMI Watchdog (for kernel debugging only)
[WARN] /proc/sys/kernel/nmi_watchdog does not exist
[INFO] Setting automatic graphics power
[WARN] Failed to set automatic graphics power: does not have switchable graphics
[INFO] Initializing with the balanced profile
setting schedutil with max 4408593
failed to set value in /sys/devices/system/cpu/cpu12/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu12/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu12/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu13/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu13/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu13/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu14/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu14/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu14/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu15/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu15/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu15/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu16/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu16/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu16/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu17/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu17/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu17/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu18/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu18/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu18/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu19/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu19/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu19/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu20/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu20/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu20/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu21/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu21/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu21/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu22/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu22/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu22/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu23/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu23/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu23/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu24/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu24/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu24/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu25/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu25/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu25/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu26/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu26/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu26/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu27/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu27/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu27/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu28/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu28/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu28/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu29/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu29/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu29/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu30/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu30/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu30/cpufreq/scaling_governor: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu31/cpufreq/scaling_min_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu31/cpufreq/scaling_max_freq: No such file or directory (os error 2)
failed to set value in /sys/devices/system/cpu/cpu31/cpufreq/scaling_governor: No such file or directory (os error 2)
[INFO] Registering dbus name com.system76.PowerDaemon
[INFO] Adding dbus path /com/system76/PowerDaemon with interface com.system76.PowerDaemon
[INFO] Handling dbus requests
[ERROR] hid_backlight: no system76_acpi::kbd_backlight led
^C[INFO] caught signal: SIGINT
[INFO] daemon exited from loop

then I went to check the service that was supposed to be running this command and it looked like this: which means the service tried running the same command as I did and got a bunch of errors so it went inactive

[nix-shell:~]$ systemctl status system76-power.service 
○ system76-power.service - System76 Power Daemon
     Loaded: loaded (/etc/systemd/system/system76-power.service; enabled; preset: enabled)
     Active: inactive (dead) since Fri 2022-12-09 15:20:40 CET; 1h 8min ago
   Duration: 3.008s
    Process: 953 ExecStart=/nix/store/ibzryww0kzswrcrsc0k9a53csz3jyhaq-system76-power-1.1.23/bin/system76-power daemon (code=exited, status=0/SUCCES>
   Main PID: 953 (code=exited, status=0/SUCCESS)
         IP: 0B in, 0B out
        CPU: 9ms

Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Registering dbus name com.system76.PowerDaemon
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Adding dbus path /com/system76/PowerDaemon with interface com.system76.PowerDaemon
Dec 09 15:20:37 beetwo systemd[1]: Started System76 Power Daemon.
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Handling dbus requests
Dec 09 15:20:37 beetwo system76-power[953]: [ERROR] hid_backlight: no system76_acpi::kbd_backlight led
Dec 09 15:20:40 beetwo system76-power[953]: [INFO] caught signal: SIGTERM
Dec 09 15:20:40 beetwo systemd[1]: Stopping System76 Power Daemon...
Dec 09 15:20:40 beetwo system76-power[953]: [INFO] daemon exited from loop
Dec 09 15:20:40 beetwo systemd[1]: system76-power.service: Deactivated successfully.
Dec 09 15:20:40 beetwo systemd[1]: Stopped System76 Power Daemon.

TLATER · December 9, 2022, 4:46pm

Can you share the output of:

journalctl --boot --unit system76-power.service > share.log

When a unit with Restart=always starts restarting it’s usually hard to see what’s going wrong.

The underlying reason to me sounds like some kind of kernel module isn’t loaded. I don’t know why. Can you share your full config, and the nixpkgs commit?

spacebuffer · December 9, 2022, 4:54pm

Didn’t think someone would care to read all of my problem, thanks!

first of all here is an interesting graph, that may help debunk this, it shows the speed of different fans on the system

the part of the graph where it was all constant, was when I wasn’t running anything (i.e the service is inactive because there are errors) so the fan speeds stayed the same, once I ran system76-power manually it seemed to get better, one fan was still constant but the other varied a little. and the sound was similar to how it was on popOS (sometimes fans are running and then they went down etc) and keep in mind running that command manually still showed a bunch of errors, though when running manually it seems to ignore them unlike when running using a service

now back to your reply, looks like the log data returned by systemd is similar to the one returned when running manually but not exactly the same

Dec 09 15:20:37 beetwo systemd[1]: Starting System76 Power Daemon...
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Starting daemon
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Connecting to dbus system bus
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Rescanning PCI bus
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] 0000:09:00.0: AMD graphics
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] 0000:09:00.0: Function for 0000:09:00.0
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] 0000:09:00.1: Function for 0000:09:00.0
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Disabling NMI Watchdog (for kernel debugging only)
Dec 09 15:20:37 beetwo system76-power[953]: [WARN] /proc/sys/kernel/nmi_watchdog does not exist
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Setting automatic graphics power
Dec 09 15:20:37 beetwo system76-power[953]: [WARN] Failed to set automatic graphics power: does not have switchable graphics
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Initializing with the balanced profile
Dec 09 15:20:37 beetwo system76-power[953]: setting schedutil with max 4408593
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu12/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu12/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu12/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu13/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu13/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu13/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu14/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu14/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu14/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu15/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu15/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu15/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu16/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu16/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu16/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu17/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu17/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu17/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu18/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu18/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu18/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu19/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu19/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu19/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu20/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu20/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu20/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu21/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu21/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu21/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu22/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu22/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu22/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu23/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu23/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu23/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu24/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu24/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu24/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu25/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu25/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu25/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu26/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu26/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu26/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu27/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu27/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu27/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu28/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu28/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu28/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu29/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu29/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu29/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu30/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu30/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu30/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu31/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu31/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: failed to set value in /sys/devices/system/cpu/cpu31/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Registering dbus name com.system76.PowerDaemon
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Adding dbus path /com/system76/PowerDaemon with interface com.system76.PowerDaemon
Dec 09 15:20:37 beetwo systemd[1]: Started System76 Power Daemon.
Dec 09 15:20:37 beetwo system76-power[953]: [INFO] Handling dbus requests
Dec 09 15:20:37 beetwo system76-power[953]: [ERROR] hid_backlight: no system76_acpi::kbd_backlight led
Dec 09 15:20:40 beetwo system76-power[953]: [INFO] caught signal: SIGTERM
Dec 09 15:20:40 beetwo systemd[1]: Stopping System76 Power Daemon...
Dec 09 15:20:40 beetwo system76-power[953]: [INFO] daemon exited from loop
Dec 09 15:20:40 beetwo systemd[1]: system76-power.service: Deactivated successfully.
Dec 09 15:20:40 beetwo systemd[1]: Stopped System76 Power Daemon.

spacebuffer · December 9, 2022, 4:57pm

also here is my full config

[spcbfr@beetwo:~]$ cat /etc/nixos/configuration.nix 
# Edit this configuration file to define what should be installed on
# your system.  Help is available in the configuration.nix(5) man page
# and in the NixOS manual (accessible by running ‘nixos-help’).

{ config, pkgs, ... }:

{
  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
    ];

  # Use the systemd-boot EFI boot loader.
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  networking.hostName = "beetwo"; # Define your hostname.
  # Pick only one of the below networking options.
  networking.networkmanager.enable = true;  # Easiest to use and most distros use this by default.
  # networking.wireless.enable = true;  # Enables wireless support via wpa_supplicant.
  programs.nm-applet.enable = true;

  # Set your time zone.
  time.timeZone = "Africa/Tunis";

  # Configure network proxy if necessary
  # networking.proxy.default = "http://user:password@proxy:port/";
  # networking.proxy.noProxy = "127.0.0.1,localhost,internal.domain";

  # Select internationalisation properties.
  i18n.defaultLocale = "en_US.UTF-8";
  console = {
    font = "Lat2-Terminus16";
    # keyMap = "us";
    useXkbConfig = true; # use xkbOptions in tty.
  };

  # Enable the X11 windowing system.
  services.xserver.enable = true;


  # Enable the GNOME Desktop Environment.
  services.xserver.displayManager.gdm.enable = true;
  services.xserver.desktopManager.gnome.enable = true;
  

  # Configure keymap in X11
  services.xserver.layout = "us";
  services.xserver.xkbOptions = "caps:escape"; # map caps to escape.


  # Enable CUPS to print documents.
  services.printing.enable = true;

  # Enable sound.
  sound.enable = true;
  hardware.pulseaudio.enable = true;

  # Enable touchpad support (enabled default in most desktopManager).
  # services.xserver.libinput.enable = true;

  # Define a user account. Don't forget to set a password with ‘passwd’.
  users.users.spcbfr = {
    isNormalUser = true;
    initialPassword = "1234";
    description = "Yusuf Bouzekri";
    extraGroups = [ "wheel" "networkmanager" ]; # Enable ‘sudo’ for the user.
    packages = with pkgs; [
      firefox
    ];
  };

  # List packages installed in system profile. To search, run:
  # $ nix search wget
  environment.systemPackages = with pkgs; [
    vim # Do not forget to add an editor to edit configuration.nix! The Nano editor is also installed by default.
    wget
  ];
  hardware.system76.enableAll = true;

  # Some programs need SUID wrappers, can be configured further or are
  # started in user sessions.
  # programs.mtr.enable = true;
  programs.gnupg.agent = {
    enable = true;
    enableSSHSupport = true;
  };

  # List services that you want to enable:

  # Enable the OpenSSH daemon.
  # services.openssh.enable = true;

  # Open ports in the firewall.
  # networking.firewall.allowedTCPPorts = [ ... ];
  # networking.firewall.allowedUDPPorts = [ ... ];
  # Or disable the firewall altogether.
  # networking.firewall.enable = false;

  # Copy the NixOS configuration file and link it from the resulting system
  # (/run/current-system/configuration.nix). This is useful in case you
  # accidentally delete configuration.nix.
  system.copySystemConfiguration = true;

  # This value determines the NixOS release from which the default
  # settings for stateful data, like file locations and database versions
  # on your system were taken. It‘s perfectly fine and recommended to leave
  # this value at the release version of the first install of this system.
  # Before changing this value read the documentation for this option
  # (e.g. man configuration.nix or on https://nixos.org/nixos/options.html).
  system.stateVersion = "22.11"; # Did you read the comment?

}

TLATER · December 9, 2022, 5:12pm

Hmm, so this makes me suspicious:

Dec 09 15:20:37 beetwo system76-power[953]: [ERROR] hid_backlight: no > system76_acpi::kbd_backlight led

The SIGTERM that kills your daemon seems to occur right after that error, so even though you probably don’t care too much about it managing the keyboard backlight it’s probably not robust to failures there?

Yet the system76 acpi module should exist by default in newer kernels.

So my suspicion is one of these:

Modern kernels don’t actually carry that module and this is a bug in that NixOS module, we should be adding some modern equivalent of system76-acpi to the external kernel modules.
system76-power has not yet been updated for modern kernels, or at least the nixpkgs version of it is out of date.
You are, in fact, on an ancient nixpkgs version, but updated your system.stateVersion erroneously by hand, and therefore anything I can see is false. Could you share the output of nix-shell -p nix-info --run "nix-info -m" just to confirm that your NixOS setup isn’t subtly broken?

If it’s not the latter, congratulations, I’m pretty sure you found a bug. The output of lsmod may also be helpful in confirming that. Next up is just finding out if the bug is in Linux, NixOS or system76-power.

For the record, I think the fan behavior changing after you run that command manually is a fluke, rather than expected behavior.

spacebuffer · December 9, 2022, 5:21pm

the returned output of the command you sent:

 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.81, NixOS, 22.11 (Raccoon), 22.11.691.5528350186a`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.0`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

I didn’t manually change the nixos version, I just installed the latest ISO and didnt touch that option at all it came set as that value

glad to here that the problem is with the keyboard and not with the cpu or gpu (which was what I initially thought) as those are very hard to replace.

just to note: I am in fact using a random chinese keyboard that I found on a local store for about 10 bucks and there is supposed to be backlight but I couldn’t for the life of me figure out how to turn it on

the output of lsmod

[spcbfr@beetwo:~]$ lsmod
Module                  Size  Used by
tls                   114688  0
rfcomm                 90112  18
qrtr                   20480  4
ns                     32768  1 qrtr
ccm                    20480  3
af_packet              57344  6
cmac                   16384  3
algif_hash             16384  1
algif_skcipher         16384  1
af_alg                 28672  6 algif_hash,algif_skcipher
bnep                   24576  2
amdgpu               7774208  15
iwlmvm                409600  0
snd_hda_codec_realtek   163840  1
snd_hda_codec_generic    98304  1 snd_hda_codec_realtek
mac80211             1003520  1 iwlmvm
ledtrig_audio          16384  1 snd_hda_codec_generic
snd_hda_codec_hdmi     73728  1
snd_hda_intel          57344  4
snd_intel_dspcfg       28672  1 snd_hda_intel
libarc4                16384  1 mac80211
snd_intel_sdw_acpi     20480  1 snd_intel_dspcfg
ip6_tables             36864  2
snd_hda_codec         176128  4 snd_hda_codec_generic,snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec_realtek
rndis_wlan             69632  0
xt_conntrack           16384  2
iommu_v2               24576  1 amdgpu
nls_iso8859_1          16384  1
edac_mce_amd           36864  0
nf_conntrack          167936  1 xt_conntrack
snd_hda_core          110592  5 snd_hda_codec_generic,snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec,snd_hda_codec_realtek
nls_cp437              20480  1
gpu_sched              45056  1 amdgpu
rndis_host             20480  1 rndis_wlan
edac_core              69632  0
cdc_ether              24576  1 rndis_host
snd_hwdep              16384  1 snd_hda_codec
vfat                   20480  1
drm_ttm_helper         16384  1 amdgpu
nf_defrag_ipv6         24576  1 nf_conntrack
intel_rapl_msr         20480  0
iwlwifi               430080  1 iwlmvm
usbnet                 57344  3 rndis_wlan,rndis_host,cdc_ether
btusb                  65536  0
nf_defrag_ipv4         16384  1 nf_conntrack
fat                    86016  1 vfat
snd_pcm               147456  4 snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec,snd_hda_core
ttm                    86016  2 amdgpu,drm_ttm_helper
intel_rapl_common      28672  1 intel_rapl_msr
btrtl                  28672  1 btusb
btbcm                  24576  1 btusb
crc32_pclmul           16384  0
drm_kms_helper        307200  1 amdgpu
gigabyte_wmi           20480  0
wmi_bmof               16384  0
btintel                45056  1 btusb
ip6t_rpfilter          16384  1
ghash_clmulni_intel    16384  0
snd_timer              49152  1 snd_pcm
input_leds             16384  0
mousedev               24576  0
evdev                  24576  11
igb                   270336  0
ipt_rpfilter           16384  1
mii                    16384  1 usbnet
mac_hid                16384  0
agpgart                40960  1 ttm
bluetooth             729088  45 btrtl,btintel,btbcm,bnep,btusb,rfcomm
cfg80211             1044480  4 rndis_wlan,iwlmvm,iwlwifi,mac80211
aesni_intel           380928  6
fb_sys_fops            16384  1 drm_kms_helper
xt_pkttype             16384  2
snd                   118784  16 snd_hda_codec_generic,snd_hda_codec_hdmi,snd_hwdep,snd_hda_intel,snd_hda_codec,snd_hda_codec_realtek,snd_timer,snd_pcm
ptp                    32768  1 igb
crypto_simd            16384  1 aesni_intel
syscopyarea            16384  1 drm_kms_helper
ecdh_generic           16384  2 bluetooth
pps_core               24576  1 ptp
sysfillrect            16384  1 drm_kms_helper
deflate                16384  1
sp5100_tco             20480  0
cryptd                 24576  3 crypto_simd,ghash_clmulni_intel
dca                    16384  1 igb
ecc                    40960  1 ecdh_generic
xt_LOG                 20480  2
sysimgblt              16384  1 drm_kms_helper
i2c_algo_bit           16384  2 igb,amdgpu
soundcore              16384  1 snd
rfkill                 36864  8 bluetooth,cfg80211
watchdog               24576  1 sp5100_tco
nf_log_syslog          20480  2
libaes                 16384  2 bluetooth,aesni_intel
rapl                   16384  0
efi_pstore             16384  0
k10temp                16384  0
i2c_piix4              28672  0
xt_tcpudp              20480  0
nft_compat             20480  11
nft_counter            16384  33
tiny_power_button      16384  0
gpio_amdpt             20480  0
gpio_generic           20480  1 gpio_amdpt
pinctrl_amd            32768  0
acpi_cpufreq           28672  0
button                 24576  0
nf_tables             266240  81 nft_compat,nft_counter
libcrc32c              16384  2 nf_conntrack,nf_tables
nfnetlink              20480  2 nft_compat,nf_tables
sch_fq_codel           20480  3
system76_io            16384  0
i8042                  32768  0
wmi                    36864  2 gigabyte_wmi,wmi_bmof
led_class              20480  3 snd_hda_codec_generic,input_leds,iwlmvm
ctr                    16384  1
atkbd                  40960  0
libps2                 20480  1 atkbd
serio                  28672  2 atkbd,i8042
loop                   40960  0
tun                    61440  0
tap                    28672  0
macvlan                28672  0
bridge                303104  0
stp                    16384  1 bridge
llc                    16384  2 bridge,stp
kvm_amd               122880  0
kvm                  1048576  1 kvm_amd
drm                   602112  13 gpu_sched,drm_kms_helper,amdgpu,drm_ttm_helper,ttm
backlight              24576  3 drm_kms_helper,amdgpu,drm
irqbypass              16384  1 kvm
i2c_core              102400  6 drm_kms_helper,i2c_algo_bit,igb,amdgpu,i2c_piix4,drm
fuse                  155648  5
pstore                 28672  2 efi_pstore
configfs               57344  1
efivarfs               16384  1
ip_tables              36864  1
x_tables               53248  9 xt_conntrack,ip6t_rpfilter,nft_compat,xt_LOG,xt_tcpudp,ip6_tables,ipt_rpfilter,xt_pkttype,ip_tables
autofs4                53248  0
ext4                  925696  1
crc32c_generic         16384  0
crc16                  16384  2 bluetooth,ext4
mbcache                16384  1 ext4
jbd2                  167936  1 ext4
hid_generic            16384  0
usbhid                 65536  0
hid                   151552  2 usbhid,hid_generic
sd_mod                 57344  4
t10_pi                 16384  1 sd_mod
crc_t10dif             20480  1 t10_pi
crct10dif_generic      16384  0
ahci                   45056  3
xhci_pci               20480  0
xhci_pci_renesas       20480  1 xhci_pci
libahci                45056  1 ahci
xhci_hcd              315392  1 xhci_pci
libata                294912  2 libahci,ahci
usbcore               331776  9 rndis_wlan,xhci_hcd,usbnet,usbhid,rndis_host,system76_io,btusb,xhci_pci,cdc_ether
scsi_mod              270336  2 sd_mod,libata
crct10dif_pclmul       16384  1
crct10dif_common       16384  3 crct10dif_generic,crc_t10dif,crct10dif_pclmul
crc32c_intel           24576  3
usb_common             16384  2 xhci_hcd,usbcore
scsi_common            16384  2 scsi_mod,libata
rtc_cmos               28672  1
dm_mod                155648  0

spacebuffer · December 9, 2022, 6:12pm

Do you think maybe I should try to use another keyboard without backlight at all

spacebuffer · December 9, 2022, 7:57pm

After a bit of investigation: I’ve found a bunch of issues on the system76-power repo, the closest of which to my case is this one Need to manually start enabled service `system76-power` on Fedora 35 Workstation (GNOME 41) after reboot (likely affects all versions at and above GNOME 41 until resolution) · Issue #299 · pop-os/system76-power · GitHub not sure what I am supposed to do now

TLATER · December 10, 2022, 2:28am

Yep, fair enough, just always worth checking you’re starting from a known good when running into baffling things like this. Sadly NixOS with channels is a little easy to get wrong.

I assumed this was a laptop with a built-in keyboard That might just be a red herring then, after all. I can see the system76-io module loaded in your lsmod output, so things are probably fine.

Good sleuthing! I wonder if this is indeed GNOME messing with power settings. The nixos-hardware modules also have a reference to this here. The NixOS gnome module enables it by default: https://github.com/NixOS/nixpkgs/blob/7a6a010c3a1d00f8470a5ca888f2f927f1860a19/nixos/modules/services/x11/desktop-managers/gnome.nix#L327

Does overriding that by setting services.power-profiles-daemon.enable to false fix your issue? If so, that’s worth contributing to nixos-hardware.

spacebuffer · December 10, 2022, 7:27am

We’re getting somewhere! I disabled the power-profiles-daemon from the config, and now the system76-power is working if I check system76-power status system76-power.service

● system76-power.service - System76 Power Daemon
     Loaded: loaded (/etc/systemd/system/system76-power.service; enabled; preset: enabled)
     Active: active (running) since Sat 2022-12-10 08:09:11 CET; 1min 3s ago
   Main PID: 934 (system76-power)
         IP: 0B in, 0B out
         IO: 3.2M read, 0B written
      Tasks: 1 (limit: 38442)
     Memory: 4.1M
        CPU: 38ms
     CGroup: /system.slice/system76-power.service
             └─934 /nix/store/ibzryww0kzswrcrsc0k9a53csz3jyhaq-system76-power-1.1.23/bin/system76-power daemon

Dec 10 08:09:11 beetwo system76-power[934]: failed to set value in /sys/devices/system/cpu/cpu30/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 10 08:09:11 beetwo system76-power[934]: failed to set value in /sys/devices/system/cpu/cpu30/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 10 08:09:11 beetwo system76-power[934]: failed to set value in /sys/devices/system/cpu/cpu31/cpufreq/scaling_min_freq: No such file or directory (os error 2)
Dec 10 08:09:11 beetwo system76-power[934]: failed to set value in /sys/devices/system/cpu/cpu31/cpufreq/scaling_max_freq: No such file or directory (os error 2)
Dec 10 08:09:11 beetwo system76-power[934]: failed to set value in /sys/devices/system/cpu/cpu31/cpufreq/scaling_governor: No such file or directory (os error 2)
Dec 10 08:09:11 beetwo system76-power[934]: [INFO] Registering dbus name com.system76.PowerDaemon
Dec 10 08:09:11 beetwo system76-power[934]: [INFO] Adding dbus path /com/system76/PowerDaemon with interface com.system76.PowerDaemon
Dec 10 08:09:11 beetwo systemd[1]: Started System76 Power Daemon.
Dec 10 08:09:11 beetwo system76-power[934]: [INFO] Handling dbus requests
Dec 10 08:09:11 beetwo system76-power[934]: [ERROR] hid_backlight: no system76_acpi::kbd_backlight led

two things are worrying me:

there are still a bunch of errors in the status log, and most of them are about important things like the cpu it seems
system76-power profile still returns ? when on popOS it returned balanced
I am not hearing any fan noise at the moment at all which is more worrying honestly than the fans running all the time (cause it can wreck my cpu if it doesn’t get enough air) here is a graph of the fan speeds after disabling power-profiles-daemon, as you can see the green fan is always working at almost a constant speed and the others are fluctuating between 0 and large speeds

image1761×902 181 KB

TLATER · December 10, 2022, 1:13pm

Yeah. I think that’s just a poor implementation. If you look at the errors, you’ll spot that it’s trying to change power settings for 32 CPUs. The reports on their github showed similar logs, so I wouldn’t be concerned about this.

It’s just setting a setting for each thread a CPU could possibly have, without checking if they exist beforehand, or checking which CPU you actually have. You seem to have 12 (0-11 don’t show errors), which means that the daemon will show 20 error messages every time it sets this. Given they also don’t check for keyboard backlight presence, they clearly don’t care about errors in the log.

In fact, you can check their source code, looks like they’re reading the maximum possible CPUs for the socket. This may be a bug.

Personally, I find it extremely odd that they have a blanket implementation like this in the first place. Setting the frequencies of all possible CPUs in all systems to the same thing seems silly if they sell different models. Different CPUs have different frequencies. Different enclosures have different temperatures. I also didn’t think you could individually address cores (let alone threads) with these settings either. Hell, this seems awful for 13th gen Intel stuff with mixed cores.

I don’t know enough about system76-power to explain why they do it this way, but clearly the computer works, so what do I know.

Is it possible there just is no default profile and someone set that setting before shipping in popOS? Apparently you can set it using e.g.

# system76-power profile balanced

Check your CPU temperature on the right instead. Tctl is the temperature the CPU reports to the system for “control” purposes, and is what your computer will use to determine if something problematic is happening.

Assuming this computer isn’t just sitting idle, the maximum it ever hit was 58°C. Your 1st die got a fair bit hotter at 66°C.

This is completely normal if the system is under load, especially in as airtight of a compartment as the thelio looks. If it gets into the 80°C+ range under load you should be a bit more concerned, though even then it isn’t universally problematic. Throttling typically starts at ~90°C for CPUs I have experience with, and recent AMD processors lock themselves in at 95°C by default, for example.

Check the CPU manufacturer’s documentation for their temperature range, if you’re concerned.

Fans running at max are incredibly much louder than fans at, say, 60%. It’s very rare for them to need to run at 100% rpm.

curriedFunctions · December 10, 2022, 2:12pm

I’ve been facing similar fan issues on my Oryx Pro model since a recent update and I need to run in “battery” profile to keep the fan from running at a pretty high speed all the time.

Here’s the output I get when I start the system76-power daemon:

[INFO] Starting daemon
[INFO] Connecting to dbus system bus
[INFO] Rescanning PCI bus
[INFO] 0000:01:00.0: NVIDIA graphics
[INFO] 0000:01:00.0: Function for 0000:01:00.0
[INFO] 0000:01:00.1: Function for 0000:01:00.0
[INFO] 0000:00:02.0: Intel graphics
[INFO] 0000:00:02.0: Function for 0000:00:02.0
[INFO] Disabling NMI Watchdog (for kernel debugging only)
[WARN] /proc/sys/kernel/nmi_watchdog does not exist
[INFO] Setting automatic graphics power
[INFO] Enabling graphics power
[INFO] Initializing with the balanced profile
[WARN] failed to iterate SCSI Host device: No such file or directory (os error 2)
setting powersave with max 4600000
[INFO] Registering dbus name com.system76.PowerDaemon
[INFO] Adding dbus path /com/system76/PowerDaemon with interface com.system76.PowerDaemon
[ERROR] fan daemon: platform hwmon not found
[INFO] Handling dbus requests
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/hid_backlight.rs:80:71
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[INFO] Setting power management to auto

When I run system76-power profile it also says “?” for the Power Profile, but if I run with sudo it says “Balanced”.

TLATER · December 10, 2022, 2:20pm

You’re definitely not running an up-to-date system76-power/NixOS, instead of the previous error your daemon just crashes. Likely the same underlying issue of GNOME 40+, though.

That, or some system settings that make your fan controls unavailable.

Mind creating a separate thread?

curriedFunctions · December 10, 2022, 3:06pm

Sure, thanks: here’s the thread