R installation within conda cannot find default libraries

Help
1

I installed conda as described in the wiki. conda-shell opens a shell, in which all conda-related commands work as expected (i.e. working with python environments works flawlessly). However, installing an R environment leads to a broken R installation: It seems that several standard-libraries cannot be found by the R interpreter:


(base) conda-shell-chrootenv:moritz@moxps:~$ conda create -n r_env r-essentials r-base
... everything installs fine ...
(base) conda-shell-chrootenv:moritz@moxps:~$ conda activate r_env
(r_env) conda-shell-chrootenv:moritz@moxps:~$ R

R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-conda_cos6-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

Error: package or namespace load failed for ‘utils’:
 .onLoad failed in loadNamespace() for 'utils', details:
  call: system(paste(which, shQuote(names[i])), intern = TRUE, ignore.stderr = TRUE)
  error: error in running command
Error: package or namespace load failed for ‘stats’:
 .onLoad failed in loadNamespace() for 'utils', details:
  call: system(paste(which, shQuote(names[i])), intern = TRUE, ignore.stderr = TRUE)
  error: error in running command
During startup - Warning messages:
1: package ‘utils’ in options("defaultPackages") was not found
2: package ‘stats’ in options("defaultPackages") was not found

I failed in finding similar errors on the web, so I suspect it’s a nixos-specific error, however I am completely out of ideas on how to find out why R cannot find these packages. utils for example seems to be present in the environment:

$ cd .conda/envs/r_env/
$ find | grep utils
...
./lib/R/library/utils/libs/utils.so
./lib/R/library/utils/misc
./lib/R/library/utils/misc/exDIF.csv
./lib/R/library/utils/misc/exDIF.dif
./lib/R/library/utils/DESCRIPTION
./lib/R/library/utils/INDEX
./lib/R/library/utils/NAMESPACE
./lib/R/library/utils/iconvlist
./lib/R/library/Rcpp/include/Rcpp/utils
./lib/R/library/Rcpp/include/Rcpp/utils/tinyformat
./lib/R/library/Rcpp/include/Rcpp/utils/tinyformat/tinyformat.h
./lib/R/library/Rcpp/include/Rcpp/utils/tinyformat.h
...

Thank you for any help on this issue

2

my experiences with conda were also very poor on nixos, I ended up resorting to using a VM (it was just a 1 or 2 time use case).

I’m not familiar with R, is there some way to convey libraries? For example, I do the following with my python venvs:

    PYTHONPATH=$PWD/$venvDir/${python.sitePackages}:$PYTHONPATH

which will make the local files available to the python interpreter

3

This is a total shot in the dark, but can you try running R after setting R_LIBS to point to where your missing R packages are stored?

4

I was hoping for such a hint. Unfortunately, setting R_LIBS to any R-associated lib directory in the environment did not change anything.

Thank you

5

Thanks for the feedback. Good to hear that I am not the only one struggling with conda.

Since I don’t use R too often, it now suffices for me to have one global R-installation and put all the packages there.

No need to mess with conda for now…

6

Shoot. There are a couple of other environment variables to try, plus the .libPaths() function which shows the full list of search paths (might be helpful for debugging).

Here’s a blog post and an Rstudio discourse thread that provide more details.

I hope this works out!

roni

7

Thank you @roni.
libPaths seems to point to the correct location. I was reading a bit about the other environment variables, but didn’t see a light. I’m trying to use the nix-version of R for now. Thank you for the support!!

> .libPaths()
[1] "/home/moritz/.conda/envs/r_env/lib/R/library"

$ find "/home/moritz/.conda/envs/r_env/lib/R/library" | grep stats
/home/moritz/.conda/envs/r_env/lib/R/library/stats
/home/moritz/.conda/envs/r_env/lib/R/library/stats/Meta
...
8

Drat!

If you do figure this out, please report back here–I’d love to know what the correct solution is for this. Good luck!

roni

9

@roni I’ve got another promising trace:

https://github.com/ContinuumIO/anaconda-issues/issues/11133

R depends on which, which might not be present in the conda R environment. Maybe it could be as simple as to add the which-derivation to conda?

Placing ‘which’ into the conda binary folder did however not fix the issue (the R console still complains about not being able to load stats and utils packages)

10

Hi, the utils package from R_HOME. this issue appeared in R 4.0 or other environments.

  1. jupyterWith/default.nix at c1ccbe1b0ee5703fd425ce0a3442e7e2ecfde352 · GTrunSec/jupyterWith · GitHub

solution: set R_HOME path

  1. R_packages path set: jupyterWith/directory.nix at c1ccbe1b0ee5703fd425ce0a3442e7e2ecfde352 · GTrunSec/jupyterWith · GitHub
11

Why do you need conda?

Have you tried to use mach-nix?

12

mach-nix looks great! Thanks for pointing me to it @igel. Unfortunately, I am using a system that requires conda (snakemake) to run the way I want it to.

@GTrunSec although it heavily looks like this is the issue for me, I still get the same error even after setting R_HOME (it was indeed unset before…):

(r_env) conda-shell-chrootenv:moritz@moxps:~/.conda/envs/r_env/bin$ R_HOME=/home/moritz/.conda/envs/r_env/lib/R ./R

Do you have a suggestion what I could still try?

Thank you both for the help!

13

@moritzschaefer Your pointer to which was a good one. Using strace -f you can find:

[pid  1144] execve("/usr/bin/which", ["/usr/bin/which", "uname"], 0x1800008 /* 177 vars */) = 0

which here reports success. I got there after adding which to conda-shell:

conda = pkgs.conda.override {                         
    extraPkgs = [ pkgs.which ];                                                                                                                                                                                                                                                
};

with this R no longer prints failures.

3 Likes
14

That is so great to hear! Do you want to start a PR for nixpkgs to get it fixed?

I added an overlay with your override, and can confirm that it works!

1 Like
15

@moritzschaefer, looks like you found a solution! Woo hoo!

roni

1 Like