Hi, is it possible to speed up a `nix copy --to file://… $store_path?
Is it possible to make nix use all cores for compression? I’ve tried the options core and max-jobs in ~/.config/nix/nix.conf. But, it looks like they behave as the documentation states and effect builds only and a copy/compression doesn’t count.
Is it possible to change the compression or its level? Haven’t found any option for this.
Thanks, Daniel
1 Like
misuzu
November 3, 2021, 4:20pm
2
Try this:
nix copy --to 'file:///tmp/cache/?compression=none&secret-key=/tmp/cache.pem' $store_path
2 Likes
bartsch
November 4, 2021, 11:13am
3
Just opened the nix code and saw that there will be an additional “parallel-compression” and “compression-level” option in the next (?) release.
#pragma once
#include "crypto.hh"
#include "store-api.hh"
#include "pool.hh"
#include <atomic>
namespace nix {
struct NarInfo;
struct BinaryCacheStoreConfig : virtual StoreConfig
{
using StoreConfig::StoreConfig;
const Setting<std::string> compression{(StoreConfig*) this, "xz", "compression", "NAR compression method ('xz', 'bzip2', 'gzip', 'zstd', or 'none')"};
const Setting<bool> writeNARListing{(StoreConfig*) this, false, "write-nar-listing", "whether to write a JSON file listing the files in each NAR"};
const Setting<bool> writeDebugInfo{(StoreConfig*) this, false, "index-debug-info", "whether to index DWARF debug info files by build ID"};
show original
NixOS:master ← dtzWill:feature/comp-level
opened 07:37PM - 05 Jul 18 UTC
As requested on #2255.
-----
Lightly tested, but could probably use a bit more… vetting :).
-----
May be useful to also add:
* [ ] Documentation
* [ ] New tests
I'll probably put together at least some test for this,
but help with documentation (what/where/how?) would be appreciated.
-----
As an aside, as I was writing the "description" for Setting's that
are part of BinaryCacheStore I wondered:
Are these flags visible somewhere to the user? I couldn't find any way
to get the store-oriented flags to be listed (with or without descriptions/values),
is this currently possible?
1 Like
nix copy was previously able to copy paths in parallel but that’s currently disabled (nix/store-api.cc at 52a3b2ee6367306a3f17fe9a2d0e2e4096852e05 · NixOS/nix · GitHub ).
1 Like
Mic92
November 11, 2021, 6:02am
6
Why was it disabled? It’s not obvious to me from skimming over the commit message: Low-latency closure copy · NixOS/nix@fe1f34f · GitHub
1 Like