This is the same issue as How to consume a `eachDefaultSystem` flake overlay? – flake-utils.lib.eachDefaultSystem will create a per-system variant of each attribute, so your overlay is actually under fpgamgr.overlay.x86_64-linux. You should see that when you run nix flake show in the library flake directory.
Overlays should generally be system-independent and should refer only to final/prev attributes and other system-independent files (e.g. local Nix files and external sources). Notably, you should not refer to specific instance of Nixpkgs.
The solution is to move the overlay and other system-independent attributes outside of the eachDefaultSystem call. And replace pkgs.callPackage with prev.callPackage. Or just import, since you are not really using the automatic passing of arguments from scope.