Can't rebuild system after deleting a package from the store

Help
1

Hi, first timer here. Couldn’t find anything relevant when googling for this.

So I’ve maybe made a terrible mistake?

I garbage collected /nix/store and forcefully deleted a package, ignoring liveness. This didn’t turn out very well because now I can’t rebuild my system…

For some background, the package I was trying to remove from the store had multiple derivations and versions that I had built locally. It is not a package published on the nix store.

nix-store --gc
# garbage collect didn't help so I tried something a bit more aggressive:
sudo nix-store delete --ignore-liveness /nix/store/*webex*

The packages were removed, yay, but it also broke something else? :frowning:

sudo nixos-rebuild switch --flake .
building the system configuration...
Traceback (most recent call last):
  File "/nix/store/i7h3374cnqxcq2956n1scaa4fqvnzspz-systemd-boot", line 274, in <module>
    main()
  File "/nix/store/i7h3374cnqxcq2956n1scaa4fqvnzspz-systemd-boot", line 237, in main
    remove_old_entries(gens)
  File "/nix/store/i7h3374cnqxcq2956n1scaa4fqvnzspz-systemd-boot", line 156, in remove_old_entries
    known_paths.append(copy_from_profile(*gen, "kernel", True))
  File "/nix/store/i7h3374cnqxcq2956n1scaa4fqvnzspz-systemd-boot", line 68, in copy_from_profile
    store_file_path = profile_path(profile, generation, name)
  File "/nix/store/i7h3374cnqxcq2956n1scaa4fqvnzspz-systemd-boot", line 64, in profile_path
    return os.readlink("%s/%s" % (system_dir(profile, generation), name))
FileNotFoundError: [Errno 2] No such file or directory: '/nix/var/nix/profiles/system-13-link/kernel'
warning: error(s) occurred while switching to the new configuration
make: *** [Makefile:3: rb-switch] Error 1

Appreciate any help on what I can do to recover. I don’t understand how deleting a package would impact kernel stuff?

2

Found this stackoverflow comment which seems kinda related? Something about broken symlinks to the store.

3

Is removing all the broken symlinks the appropriate action to take? Are these symlinks to different OS generations that now are gone because I deleted a package that were included in them?

johanan@voidm ~/os for f in $(find /nix/var/nix/profiles/); do if [ ! -e "$f" ]; then echo "$f is broken"; fi; done
/nix/var/nix/profiles/system-80-link is broken
/nix/var/nix/profiles/system-99-link is broken
/nix/var/nix/profiles/system-19-link is broken
/nix/var/nix/profiles/system-27-link is broken
/nix/var/nix/profiles/system-85-link is broken
/nix/var/nix/profiles/system-38-link is broken
/nix/var/nix/profiles/system-106-link is broken
/nix/var/nix/profiles/system-16-link is broken
/nix/var/nix/profiles/system-35-link is broken
/nix/var/nix/profiles/system-117-link is broken
/nix/var/nix/profiles/system-87-link is broken
/nix/var/nix/profiles/system-17-link is broken
/nix/var/nix/profiles/system-93-link is broken
/nix/var/nix/profiles/system-89-link is broken
/nix/var/nix/profiles/system-25-link is broken
/nix/var/nix/profiles/system-90-link is broken
/nix/var/nix/profiles/system-32-link is broken
/nix/var/nix/profiles/system-14-link is broken
/nix/var/nix/profiles/system-96-link is broken
/nix/var/nix/profiles/system-92-link is broken
/nix/var/nix/profiles/system-31-link is broken
/nix/var/nix/profiles/system-13-link is broken
/nix/var/nix/profiles/system-88-link is broken
/nix/var/nix/profiles/system-77-link is broken
/nix/var/nix/profiles/system-39-link is broken
/nix/var/nix/profiles/system-22-link is broken
/nix/var/nix/profiles/system-83-link is broken
/nix/var/nix/profiles/system-103-link is broken
/nix/var/nix/profiles/system-40-link is broken
/nix/var/nix/profiles/system-78-link is broken
/nix/var/nix/profiles/system-28-link is broken
/nix/var/nix/profiles/system-104-link is broken
/nix/var/nix/profiles/system-91-link is broken
/nix/var/nix/profiles/system-112-link is broken
/nix/var/nix/profiles/system-116-link is broken
/nix/var/nix/profiles/system-102-link is broken
/nix/var/nix/profiles/system-95-link is broken
/nix/var/nix/profiles/system-33-link is broken
/nix/var/nix/profiles/system-113-link is broken
/nix/var/nix/profiles/system-94-link is broken
/nix/var/nix/profiles/system-97-link is broken
/nix/var/nix/profiles/system-41-link is broken
/nix/var/nix/profiles/system-110-link is broken
/nix/var/nix/profiles/system-18-link is broken
/nix/var/nix/profiles/system-20-link is broken
/nix/var/nix/profiles/system-29-link is broken
/nix/var/nix/profiles/system-107-link is broken
/nix/var/nix/profiles/system-82-link is broken
/nix/var/nix/profiles/system-98-link is broken
/nix/var/nix/profiles/system-81-link is broken
/nix/var/nix/profiles/system-24-link is broken
/nix/var/nix/profiles/system-36-link is broken
/nix/var/nix/profiles/system-108-link is broken
/nix/var/nix/profiles/system-101-link is broken
/nix/var/nix/profiles/system-26-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-77-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-55-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-15-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-59-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-78-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-57-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-11-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-54-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-61-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-7-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-9-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-17-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-19-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-66-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-63-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-13-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-72-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-9-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-56-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-7-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-76-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-73-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-48-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-79-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-14-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-62-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-15-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-52-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-70-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-8-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-71-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-58-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-18-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-51-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-49-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-64-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-65-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-14-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-12-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-18-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-17-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-74-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-12-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-8-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-67-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-11-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-16-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-50-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-10-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-53-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-16-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-60-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-68-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-10-link is broken
/nix/var/nix/profiles/per-user/johanan/profile-13-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-69-link is broken
/nix/var/nix/profiles/per-user/johanan/home-manager-75-link is broken
/nix/var/nix/profiles/system-30-link is broken
/nix/var/nix/profiles/system-105-link is broken
/nix/var/nix/profiles/system-21-link is broken
/nix/var/nix/profiles/system-114-link is broken
/nix/var/nix/profiles/system-42-link is broken
/nix/var/nix/profiles/system-111-link is broken
/nix/var/nix/profiles/system-118-link is broken
/nix/var/nix/profiles/system-109-link is broken
/nix/var/nix/profiles/system-23-link is broken
/nix/var/nix/profiles/system-84-link is broken
/nix/var/nix/profiles/system-34-link is broken
/nix/var/nix/profiles/system-15-link is broken
/nix/var/nix/profiles/system-76-link is broken
/nix/var/nix/profiles/system-115-link is broken
/nix/var/nix/profiles/system-86-link is broken
/nix/var/nix/profiles/system-37-link is broken
/nix/var/nix/profiles/system-79-link is broken
/nix/var/nix/profiles/system-100-link is broken
johanan@voidm ~/os
4

I think you’ll need to run sudo nix-store --verify --check-contents --repair.

There are lighter alternatives, but your store is very broken, so this is just less effort. Deleting files from it by hand is always a bad idea.

What were you trying to achieve by doing this? Maybe I can point you to the “right” way for the future :slight_smile:

5

Thanks for the reply!

For this particular application I was just trying to get it to show up in the Gnome applications menu lol… I figured that it could have something to do with there being multiple versions of the app in the store and that the system didn’t know which one to pick? All of the versions in the store had a .desktop entry, and I did see it show up some months back when I installed it the first time.

I’ll try your suggestion, thank you. Curious to know, how did this break my store? I figured that it was safe to do because it was a local package installed by me, with no packages depending on it.

6

Have the *.desktop entry in $out/share/applications and install the package through configuration.nix, home-manager or nix-env.

As long as the contents of your derivation end up beeing linked in either /run/current-system/sw or ~/.nix-profile gnome (or any other launcher) should pick the desktop files up eventually.

Especially the launchers of the bigger DEs require some help, by flushing their caches, I do not know how exactly thats done, though at least nixos-rebuild/configuration.nix should run the cache-flush on activation if it is aware of your DE. (At least this has been told to me).

If in doubt, log out and log back in.

1 Like
7

Yeah, I did install it via home-manager:

    (pkgs.callPackage ../local-pkgs/webex.nix {})

and it does have the .desktop file in the correct path:

[nix-shell:~/os/local-pkgs/result]$ tree share
share
└── applications
    └── webex.desktop

1 directory, 1 file

[nix-shell:~/os/local-pkgs/result]$

I think a possible cause could have been that the URL that fetches the .deb file is not versioned, so I might have installed packages with different hashes but with an unchanged version. Not sure if that makes a difference?

As for sudo nix-store --verify --check-contents --repair, it did execute successfully but it did not fix the error when trying to rebuild.

Should I just delete all the broken symlinks?

8

All right, so I backed up the profiles with cp -r -d /nix/var/nix/profiles . and then removed all the broken symlinks:

for f in $(find /nix/var/nix/profiles/); do if [ ! -e "$f" ]; then sudo rm $f; fi; done

Rebuilding the system now works.

So reading up on profiles, here’s what I think happened.

The store did not break, but I had lots of generations that broke because I forcefully removed a package that they still referenced.

Thanks for the help all! Really appreciate the quick responses :slight_smile:

edit: for the .desktop entry not showing up in Gnome, I think it’s due to a change in the package so not Nix-related.

9

Nice that you’ve figured it out :slight_smile:

For reference, what you may have run into is that old versions of the package persisted in old generations. You can use -d on nix-collect-garbage to flush those in most cases.

1 Like