Pretty new to nix, but I’ve been reading through as much as I can find, but I’m stuck.
I am wanting to include a library that I have built from source using an absolute file path.
I have a custom library I for work that I need accessible globally to other packages (IDEs, etc). The source lives in a repo (in Azure) that can only be accessed when on a VPN, and I really cannot have to sign into a work VPN every time I want to rebuild my config.
I’m currently trying to use nix-ld to set my paths, but I cannot figure it out since I don’t have a package built for it. It would be great if I could either
A) be able to set a global path to include /path/to/my/lib - or -
B) be able to create some package that I could build once while on the VPN, but then as long as I didn’t need to update it would not have it interfere with other changes while not connected
I’m on version 23.11
What am I missing, and how can I get there? Thank you, all
I think what you are suggesting should be possible somehow but I do not think it is the Nix way™. Personally, I would start by creating a Nix derivation for the library.
I would probably just use local checkout of the library repo as the source for the library derivation. Then VPN would only be needed to update the repo. If I wanted it more self-contained, I could even add the library repo as a subtree to my configuration repo.
Alternately, you could symlink the src to the library derivation output somewhere. That way, the source will not be garbage collected while the library package is accessible from some GC root. (A sort of poor man’s system.includeBuildDependencies, when that is too heavy.)
2 Likes
Thank you for the advice! I’m still struggling with the actual script to create the derivation. This is the latest iteration, but I’ve looked through examples from multiple docs and tutorials
What I have now is this, which is failing with a new error. Any idea what I’m doing wrong?:
error: infinite recursion encountered
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:506:28:
505| builtins.addErrorContext (context name)
506| (args.${name} or config._module.args.${name})
| ^
507| ) (lib.functionArgs f);
my_repo.nix
{ config, lib, stdenv, pkgs, cmake, makeWrapper, ... }:
let
build_pkgs = with pkgs; [
pkgs.gcc
pkgs.gdb
pkgs.python3
pkgs.cmake
pkgs.sqlite.dev
pkgs.libtiff.dev
pkgs.curl.dev
];
in
stdenv.mkDerivation { # removed "rec" to prevent infinite recursion
pname = "my_project";
version = "1.0.0";
src = /path/to/repo/.;
nativeBuildInputs = build_pkgs;
LD_LIBRARY_PATH = "${pkgs.lib.makeLibraryPath build_pkgs}:${pkgs.stdenv.cc.cc.lib}/lib";
SQLITE3_LIBRARY = "${pkgs.sqlite.dev}/lib/libsqlite3.so"; #
SQLITE3_INCLUDE_DIR = "${pkgs.sqlite.dev}/include";
TIFF_LIBRARY = "${pkgs.libtiff.dev}/lib/libtiff.so";
TIFF_INCLUDE_DIR = "${pkgs.libtiff.dev}/include";
CURL_LIBRARY = "${pkgs.curl.dev}/lib";
CURL_INCLUDE_DIR = "${pkgs.curl.dev}/include";
# Configure phase
configurePhase = ''
mkdir $src/build
cd $src/build
cmake ..
''; # Does it need "-DCMAKE_INSTALL_PREFIX=$out"" instead?
# Build phase
buildPhase = ''
cd $src/build
cmake --build .
'';
# Install phase
installPhase = ''
mkdir -p $out/bin
mv $src/bin/* $out/bin/
mkdir -p $out/lib
mv $src/lib/* $out/lib/
'';
meta = with lib; {
description = "Customized version of proj";
homepage = "https://example.com/my_project";
license = licenses.mit;
maintainers = with maintainers; [ "Names" ];
};
}
This is a bit sparse to be able to tell anything.
How are you invoking my_repo.nix? And what is the full error with --show-trace?
I’m including the .nix file in my main configuration.nix file:
imports = [
./hardware-configuration.nix
/home/myself/.dotfiles/my_repo.nix
];
Although now I’m beginning to think that’s the the correct method to include it.
The errors with --show-trace seem repeat this same pattern until it fails:
error: infinite recursion encountered
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:506:28:
505| builtins.addErrorContext (context name)
506| (args.${name} or config._module.args.${name})
| ^
507| ) (lib.functionArgs f);
building the system configuration...
error:
… while evaluating the attribute 'config.system.build.toplevel'
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:320:9:
319| options = checked options;
320| config = checked (removeAttrs config [ "_module" ]);
| ^
321| _module = checked (config._module);
… while calling the 'seq' builtin
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:320:18:
319| options = checked options;
320| config = checked (removeAttrs config [ "_module" ]);
| ^
321| _module = checked (config._module);
… while evaluating a branch condition
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:261:9:
260| checkUnmatched =
261| if config._module.check && config._module.freeformType == null && merged.unmatchedDefns != [] then
| ^
262| let
… in the left operand of the AND (&&) operator
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:261:72:
260| checkUnmatched =
261| if config._module.check && config._module.freeformType == null && merged.unmatchedDefns != [] then
| ^
262| let
… in the left operand of the AND (&&) operator
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:261:33:
260| checkUnmatched =
261| if config._module.check && config._module.freeformType == null && merged.unmatchedDefns != [] then
| ^
262| let
… while evaluating a branch condition
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:254:12:
253|
254| in if declaredConfig._module.freeformType == null then declaredConfig
| ^
255| # Because all definitions that had an associated option ended in
… from call site
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:242:28:
241| # For definitions that have an associated option
242| declaredConfig = mapAttrsRecursiveCond (v: ! isOption v) (_: v: v.value) options;
| ^
243|
… while calling 'mapAttrsRecursiveCond'
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/attrsets.nix:696:5:
695| # Attribute set to recursively map over.
696| set:
| ^
697| let
… from call site
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:234:33:
233| ({ inherit lib options config specialArgs; } // specialArgs);
234| in mergeModules prefix (reverseList collected);
| ^
235|
… while calling 'reverseList'
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/lists.nix:510:17:
509| */
510| reverseList = xs:
| ^
511| let l = length xs; in genList (n: elemAt xs (l - n - 1)) l;
… from call site
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:229:25:
228| merged =
229| let collected = collectModules
| ^
230| class
… while calling anonymous lambda
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:443:37:
442|
443| in modulesPath: initialModules: args:
| ^
444| filterModules modulesPath (collectStructuredModules unknownModule "" initialModules args);
… from call site
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:444:7:
443| in modulesPath: initialModules: args:
444| filterModules modulesPath (collectStructuredModules unknownModule "" initialModules args);
| ^
445|
… while calling 'filterModules'
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:411:36:
410| # modules recursively. It returns the final list of unique-by-key modules
411| filterModules = modulesPath: { disabled, modules }:
| ^
412| let
… while calling anonymous lambda
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:437:31:
436| disabledKeys = concatMap ({ file, disabled }: map (moduleKey file) disabled) disabled;
437| keyFilter = filter (attrs: ! elem attrs.key disabledKeys);
| ^
438| in map (attrs: attrs.module) (builtins.genericClosure {
… from call site
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:398:22:
397| let
398| module = checkModule (loadModule args parentFile "${parentKey}:anon-${toString n}" x);
| ^
399| collectedImports = collectStructuredModules module._file module.key module.imports args;
… while calling anonymous lambda
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:357:11:
356| then
357| m:
| ^
358| if m._class != null -> m._class == class
… from call site
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:398:35:
397| let
398| module = checkModule (loadModule args parentFile "${parentKey}:anon-${toString n}" x);
| ^
399| collectedImports = collectStructuredModules module._file module.key module.imports args;
… while calling 'loadModule'
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:334:53:
333| # Like unifyModuleSyntax, but also imports paths and calls functions if necessary
334| loadModule = args: fallbackFile: fallbackKey: m:
| ^
335| if isFunction m then
… from call site
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:352:14:
351| throw "Module imports can't be nested lists. Perhaps you meant to remove one level of lists? Definitions: ${showDefs defs}"
352| else unifyModuleSyntax (toString m) (toString m) (applyModuleArgsIfFunction (toString m) (import m) args);
| ^
353|
… while calling 'unifyModuleSyntax'
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:452:34:
451| of ‘options’, ‘config’ and ‘imports’ attributes. */
452| unifyModuleSyntax = file: key: m:
| ^
453| let
… from call site
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:352:59:
351| throw "Module imports can't be nested lists. Perhaps you meant to remove one level of lists? Definitions: ${showDefs defs}"
352| else unifyModuleSyntax (toString m) (toString m) (applyModuleArgsIfFunction (toString m) (import m) args);
| ^
353|
… while calling 'applyModuleArgsIfFunction'
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:486:39:
485|
486| applyModuleArgsIfFunction = key: f: args@{ config, options, lib, ... }:
| ^
487| if isFunction f then applyModuleArgs key f args else f;
… from call site
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:487:26:
486| applyModuleArgsIfFunction = key: f: args@{ config, options, lib, ... }:
487| if isFunction f then applyModuleArgs key f args else f;
| ^
488|
… while calling 'applyModuleArgs'
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:489:29:
488|
489| applyModuleArgs = key: f: args@{ config, options, lib, ... }:
| ^
490| let
… from call site
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:513:8:
512| # works.
513| in f (args // extraArgs);
| ^
514|
… while calling anonymous lambda
at /home/myself/.dotfiles/my_repo.nix:1:1:
1| { config, lib, stdenv, pkgs, cmake, makeWrapper, ... }:
| ^
2|
… while calling anonymous lambda
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:504:44:
503| context = name: ''while evaluating the module argument `${name}' in "${key}":'';
504| extraArgs = builtins.mapAttrs (name: _:
| ^
505| builtins.addErrorContext (context name)
… while evaluating the module argument `stdenv' in "/home/myself/.dotfiles/my_repo.nix":
error: infinite recursion encountered
at /nix/store/1xclxik35hm5agkgp9a0wh6465an2j98-nixos-23.11/nixos/lib/modules.nix:506:28:
505| builtins.addErrorContext (context name)
506| (args.${name} or config._module.args.${name})
| ^
507| ) (lib.functionArgs f);
Ah. imports expects NixOS modules, whereas my_repo.nix is a Nixpkgs package expression.
You should be able to instantiate the package with pkgs.callPackage, which will fill in the dependency arguments.
But for now, including it to your system configuration is probably too early. You should first try to make the package itself build and you can do that directly with nix-build --expr 'let pkgs = import <nixpkgs> {}; in pkgs.callPackage ~/.dotfiles/my_repo.nix { }' and check the result/ directory symlink. That will allow for faster iteration (no need to rebuild the whole system).
After you are happy with it, you would create a package for a program that depends on the library. You should be able to build it similarly:
nix-build --expr 'let pkgs = import <nixpkgs> {}; my_library = pkgs.callPackage ~/.dotfiles/my_repo.nix { }; in pkgs.callPackage ~/.dotfiles/my_program.nix { inherit my_library; }'
Finally, when everything is ready, you can use it in your configuration.nix as follows:
{ pkgs, … }:
let
# And now, you will be able to use my_lib variable in the rest of the file.
my_lib = pkgs.callPackage ~/.dotfiles/my_repo.nix { };
# For example, pass it to another package that depends on it.
my_program = pkgs.callPackage = ~/.dotfiles/my_program.nix {
inherit my_lib;
};
in
{
imports = [
./hardware-configuration.nix
];
environment.systemPackages = [
# You can install programs through this. Note that installing libraries globally is not generally supported:
# https://wiki.nixos.org/wiki/FAQ#I_installed_a_library_but_my_compiler_is_not_finding_it._Why?
my_program
];
# …
}
1 Like
Thank you! I’ve gotten the package to build and install cleanly into the nix store, and is now included in my main configuration. So now I guess the lingering question is what is the correct nix way to make those libraries available to my system?
I have in my configuration:
nixpkgs.config.packageOverrides = pkgs: {
my-repo = pkgs.callPackage /home/myself/.dotfiles/my_repo.nix { };
};
programs.nix-ld.enable = true;
programs.nix-ld.libraries = [
pkgs.my-repo
];
But I’m still unable to locate the libraries. For example: Running code inside of Rider that needs to access that installed lib. Or running a dotnet program from the console that is looking for that library.