# Using NixOS in an isolated environment
**URL:** https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369
**Category:** Help
**Created:** [July 3, 2019, 9:05pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369 "2019-07-03T21:05:22Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@ejpcmac](https://discourse.nixos.org/u/ejpcmac)
#### Post date: [July 3, 2019, 9:05pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/1 "2019-07-03T21:05:22Z")
Hi! I will work on a scientific base on Kerguelen Islands for a full year from next November. There, internet access is provided through a low-bandwidth VSAT connection and reserved for professional activities, so I won’t be able to use it to install software for my personal projects.
I want to use this constraint as an opportunity to study the use of NixOS in an isolated environment. My goal is to be able to update my configuration, install new software and deploy NixOS machines without relying on an internet access.
## What I already know
So far, I know that setting up a local NixOS channel is as easy as getting `https://nixos.org/channels/nixos-19.03/nixexprs.tar.xz`. I can also copy the corresponding binary cache locally by running:
```auto
curl -L https://nixos.org/channels/nixos-19.03/store-paths.xz \
| xz -d \
| xargs \
nix copy --store https://cache.nixos.org/ \
--to file:///path/to/nixos-19.03/
```
Now, that works great if I want to install something that _is_ in the binary cache. And here start my questions.
## What I want to know
1. **What exactly is included in the binary cache for a full channel, like `nixos-19.03`?**
2. **For what is not included, are the sources cached?**
3. **How can I cache the full set of sources?**
4. **What about the different architectures?**
Any insight on these subjects is well appreciated 🙂
---
### Author:  [@markuskowa](https://discourse.nixos.org/u/markuskowa)
#### Post date: [July 4, 2019, 11:23am UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/2 "2019-07-04T11:23:20Z")
I would be really interested in how big that cache would be (without sources and with sources).
The sources are also just derivations and as such they are also cached. One possible starting point could be to write a nix functions that walks over all derivations/attributes in the package set and and extracts `src.outPath` from each of them. You can then create a list and feed that to your `nix copy` solution above. However, this might miss some `fetch*` statements that are not directly placed as `src` attributes.
---
### Author:  [@marius851000](https://discourse.nixos.org/u/marius851000)
#### Post date: [July 4, 2019, 12:09pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/3 "2019-07-04T12:09:00Z")
Another possibility may also be to generate a list of package that nixpkgs contain, and then, for each package, evaluate the package (without compiling, with readonly), with a modified fetchurl function that output the source and the hash of the url. You can then fetch these path with a script.
---
### Author:  [@ejpcmac](https://discourse.nixos.org/u/ejpcmac)
#### Post date: [July 4, 2019, 6:46pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/4 "2019-07-04T18:46:28Z")
> [@markuskowa](#):
>
> I would be really interested in how big that cache would be (without sources and with sources).
I’ve finished simple Nix copies for `nixos-19.03-small` and `nixos-19.03`, the sizes are here, showing it is not _that_ big:
```auto
NAME REFER COMPRESS RATIO
helios/test/cache 1,11M lz4 1.01x
helios/test/cache/nixos-19.03 71,0G lz4 1.01x
helios/test/cache/nixos-19.03-small 572M lz4 1.02x
```
However, I don’t know (yet) the proportion of what is not included.
@tilpner sent me some scripts on #nix:matrix.org, and notably [this repository](https://github.com/NixOS/nixos-channel-scripts). I’ll dig into it, maybe there are some interesting things there.
---
### Author:  [@markuskowa](https://discourse.nixos.org/u/markuskowa)
#### Post date: [July 5, 2019, 9:11am UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/5 "2019-07-05T09:11:08Z")
😮 71 GB sounds pretty small but those are then all xz compressed nar files. Looking at the output paths this looks like it is only final build products but no sources (which should be sufficient to install packages on a running system?).
---
### Author:  [@xaverdh](https://discourse.nixos.org/u/xaverdh)
#### Post date: [August 9, 2019, 11:07am UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/6 "2019-08-09T11:07:42Z")
I am actually in a similar situation (want to set up a local cache for a location with horrible internet connection) . I wondered if I should mirror [tarballs.nixos.org](http://tarballs.nixos.org) as well. What would be the preferred method for that?
---
### Author:  [@ejpcmac](https://discourse.nixos.org/u/ejpcmac)
#### Post date: [August 10, 2019, 8:18am UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/7 "2019-08-10T08:18:28Z")
I think it would be good to have one indeed. The full [tarballs.nixos.org](http://tarballs.nixos.org) must be really big though: what we need is only the tarballs for a given version of `nixpkgs`. For this, we want to get all the external files referenced by all the derivations we want to cache, then download them.
I haven’t worked on it lately as I was on vacation, and now I still need to do a lot of other things. I try to advance on this subject in parallel to the other parts of my preparation.
The blocking points for me right now are:
1. For a given derivation, how do we know its external dependencies?
2. What is the set of derivations that we want to include? For instance, many packages in `texlive.combined.scheme-full` are not in the binary cache, but this is typically something that would be interesting to have in the tarballs. Same for `beam.packages.erlangRxx.*`: not everything is pre-compiled with every erlang version available here, but I’d like to have the sources so I can compile them offline.
---
### Author:  [@7c6f434c](https://discourse.nixos.org/u/7c6f434c)
#### Post date: [August 10, 2019, 11:34am UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/8 "2019-08-10T11:34:32Z")
> 1. For a given derivation, how do we know its external dependencies?
I would do a `nix-store -q -R` to find all the dependencies, then do `nix-store -q -b outputHash` to check which ones are fixed-output. Hopefully Nixpkgs checkout + all fixed-output dependencies should be enough.
> 1. What is the set of derivations that we want to include? For instance, many packages in `texlive.combined.scheme-full` are not in the binary cache, but this is typically something that would be interesting to have in the tarballs. Same for `beam.packages.erlangRxx.*`: not everything is pre-compiled with every erlang version available here, but I’d like to have the sources so I can compile them offline.
I guess you would need to write a more aggressive version of release.nix… Basically, you `tryEval` the Nixpkgs (this should succeed) and find out all the attribute names in the package set. Then for every attribute you `tryEval` it and check if there is `outPath` (if yes: one more derivation found, most likely) and also list all its subattributes to check later. Rinse, repeat. Of course you’d better check that the data values are attribute sets before recursion.
---
### Author:  [@volth](https://discourse.nixos.org/u/volth)
#### Post date: [August 10, 2019, 10:24pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/9 "2019-08-10T22:24:27Z")
> [@7c6f434c](#):
>
> I would do a `nix-store -q -R` to find all the dependencies, then do `nix-store -q -b outputHash` to check which ones are fixed-output. Hopefully Nixpkgs checkout + all fixed-output dependencies should be enough.
Until one will need a new package, for example to set up a VPN to get access to the network (I guess it is exactly the case of recent post of [Rebuild NixOS Offline](https://discourse.nixos.org/t/rebuild-nixos-offline/3679) as I hit this problem few times as well).
Finding ALL possible fixed-output derivations (even not included to release.nix) looks doable. Challenging but doable.
And besides solving the problem with isolated environments it would be useful to have a bot to check if fixed-output derivations still have valid hash.
(I bet that `fetchurl { url = http://download.processing.org/reference.zip; sha256 = ... }` does not, it is updated monthly. And there must be many other like this. It would be useful to have a list of problem spots, published on Hydra or ryatm-bot)
---
### Author:  [@markuskowa](https://discourse.nixos.org/u/markuskowa)
#### Post date: [August 11, 2019, 10:25pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/10 "2019-08-11T22:25:27Z")
Adding here to the idea here to use tryEval over nixpkgs:
1. use nix-instantiate to then instantiate all derivations. This should give a list of all relevant derivations.
2. iterate with `nix-store -qR` over all drv files to get all input/dependency paths (which includes all sources).
3. Download all of the store paths from step 2 with `nix-store --realise` from the cache.
That should put all derivations in the store path to rebuild all packages offline(?)
---
### Author:  [@deliciouslytyped](https://discourse.nixos.org/u/deliciouslytyped)
#### Post date: [August 12, 2019, 2:07am UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/11 "2019-08-12T02:07:48Z")
I’m told the sum total of derivations in the caches it 180TB. It’s never been garbage collected. 😃
Don’t forget to have a checkout of the nixpkgs repository too!
---
### Author:  [@7c6f434c](https://discourse.nixos.org/u/7c6f434c)
#### Post date: [August 12, 2019, 5:08am UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/12 "2019-08-12T05:08:45Z")
> 1. iterate with `nix-store -qR` over all drv files to get all input/dependency paths (which includes all sources).
> 2. Download all of the store paths from step 2 with `nix-store --realise` from the cache.
Erm, you really want to select only the fixed-output derivations — there are quite a few long-build packages that are also not in the cache.
Actually, it is a good question whether one wants to include the sources of previously-working broken packages…
---
### Author:  [@volth](https://discourse.nixos.org/u/volth)
#### Post date: [August 12, 2019, 10:07am UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/13 "2019-08-12T10:07:57Z")
> [@markuskowa](#):
>
> 1. use nix-instantiate to then instantiate all derivations.
I am afraid a single `nix-instantiate` run would not be enough, some FOD may be excluded by
`if stdenv.isAarch64 then fetchpatch {}`
or
`if config.use_pulseaudio then fetchpatch {}`
The task is closer to test coverage or fuzzers
---
### Author:  [@markuskowa](https://discourse.nixos.org/u/markuskowa)
#### Post date: [August 12, 2019, 7:57pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/14 "2019-08-12T19:57:26Z")
That would address one the original questions: how do you get everything into your store so that you can re-build a random package of your choice? For that I need all FODs, right?
---
### Author:  [@markuskowa](https://discourse.nixos.org/u/markuskowa)
#### Post date: [August 12, 2019, 8:02pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/15 "2019-08-12T20:02:15Z")
OK, so even this one may still miss some FODs.
---
### Author:  [@deliciouslytyped](https://discourse.nixos.org/u/deliciouslytyped)
#### Post date: [August 29, 2019, 12:48pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/16 "2019-08-29T12:48:44Z")
sidenote: this could be useful in high security contexts where you don’t want to ever connect a machine to the network?
---
### Author:  [@deliciouslytyped](https://discourse.nixos.org/u/deliciouslytyped)
#### Post date: [May 31, 2020, 7:43am UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/17 "2020-05-31T07:43:04Z")
So what’s the status here currently?
---
### Author:  [@AleXoundOS](https://discourse.nixos.org/u/AleXoundOS)
#### Post date: [May 31, 2020, 1:10pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/18 "2020-05-31T13:10:20Z")
I’m working hard on a [specific mirroring tool](https://github.com/AleXoundOS/nix-mirror). It’s not ready yet and there is no 100% certainty it solves the task fully. For example, [this comment](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/13) makes me think it’s far from done. Currently, the README does not the reflect the current state, there were lots of changes since its last update! Also, mirroring [https://tarballs.nixos.org](https://tarballs.nixos.org) is no more something the project is focused on.
---
### Author:  [@ejpcmac](https://discourse.nixos.org/u/ejpcmac)
#### Post date: [June 28, 2020, 12:56pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/19 "2020-06-28T12:56:17Z")
Hello everyone!
After more than six months using NixOS in an isolated environment, let me share my experience and new questions.
## Initial setup
Last november, I have created a mirror for the current `nixos-19.09` channel, and nix-copied the associated store paths to build a local binary cache. This was achieved using this script:
```sh
#!/bin/sh
set -e
if [$# -ne 3]; then
echo "usage: $0 "
exit 1
fi
channel=https://nixos.org/channels/$1
cache=$(curl -sSL $channel/binary-cache-url)
mirror=$2
mirror_cache=$3
printf "\n\e[32m=> Create a local channel in $mirror...\e[0m\n\n"
# Build the local channel.
mkdir -p "$mirror"
curl -L $channel/git-revision > "$mirror/git-revision"
curl -L $channel/nixexprs.tar.xz > "$mirror/nixexprs.tar.xz"
curl -L $channel/store-paths.xz > "$mirror/store-paths.xz"
printf "file://$mirror_cache" > "$mirror/binary-cache-url"
printf "\n\e[32m=> Copy the binary cache to $mirror_cache...\e[0m\n\n"
# Build the local binary cache.
mkdir -p "$mirror_cache"
xzcat "$mirror/store-paths.xz" | xargs nix copy --store $cache --to "file://$mirror_cache"
printf "\n\e[32m\e[1mThe mirror is up to date!\e[0m\n\n"
```
Once this mirror has been ready on my machine, I have switched my `nixos` channel to it and added this to my `configuration.nix`:
```nix
# Nix configuration for offline mode.
nix = {
binaryCaches = lib.mkForce ["file:///data/Mirroirs/nixpkgs/cache"];
gc.automatic = false;
};
```
Then, I rebuilt my system, and was ready to go offline for a year.
## What works out of the box
### Everyday software installation on an already-working system
Each time I have needed new software on my machine that is already present in nixpkgs, rebuilding my system using the local binary cache worked smoothly. I’m sure there are some edge cases where an internet access would be required, but I have not encountered any of them yet.
### New server-like system setup
Once arrived in Kerguelen, I have copied my local mirror to a server in our network to make it easily available in the network. Installing a NixOS server instance for test purpose was pretty straightforward: I just needed to configure the channel and substituters in the installer, and all worked as expected.
## When things are missing
Things have become more complicated since I wanted to setup machines with a desktop environment: some store paths are missing from the local binary cache. Fore some reason, Nix wants to rebuild some things from source, then need the sources and patches for OpenSSL, Gettext and other few software.
As a workaround, I use `nix-serve` to share my Nix store, which contains the missing paths, allowing me to deploy new desktop machines, but this feels highly hacky.
I have then a few questions regarding these missing paths:
1. How to get a list of what store paths are needed by a configuration but are missing from a given binary cache?
2. Why are some comonly-needed store paths missing from the `nixos-xx.xx` channel? I mean, a configuration with a desktop environment is quite common, and I don’t remember building anything from source when connected to the internet: that means the missing paths do exist in `cache.nixos.org` somehow, but are not part of a channel. Is this possible?
Before to go any further, I would like to be able to create a binary cache containing everything needed to setup new « standard » desktop machines. Currently _almost_ everything is there, but these tiny missing things are a blocker.
Another case that needs my Nix store as a complementary binary cache is the setup of the first generation for `home-manager`. I can setup a local mirror for its channel so I can install it on machines without internet, but some things are missing to build the _first_ generation. Afterwards all works OK with my incomplete binary cache.
## System upgrades
I am able to get new versions of nixpkgs and my binary cache mirror every few months, when the Marion Dufresne brings us postal mail. Last time was in April, next time will be in September, but I need to prepare everything for the parcel to be shipped from Metropolitan France before July, 15th. I have access to a distant NixOS machine through our very slow and high latency VSAT link, from which I am able to build new versions of my binary cache.
Provided that my initial binary cache was incomplete, I have done several things when preparing for my first update in February:
1. I have created a mirror for the new version of NixOS 19.09 available, as incomplete as the first one,
2. I have built my configuration on the distant NixOS machine, using the mirror channel, and exported its output closure:
3. To be sure I would not miss anything, I even built a full cache, including the build derivations:
4. I repeated this for my `home-manager` configuration, and some custom `default.nix` files for development projects environments.
In April, when the SSD containing the new mirror and these store exports arrived in Kerguelen, I first tried to rebuild my system using only the updated local binary cache. Thing where missing, as expected, so I did a `sudo nix-store --import < path/to/export.deps` and everything worked as expected.
This ability to build a configuration somewhere, export it and import it in some other store kind of amazes me. This is a really powerful feature of Nix. However, this is still a workaround for my problem, and it works only when you know in advance which configurations are present in the isolated environment.
I’d prefer to build a binary cache that already contains what is needed for most desktop configurations, and try this enhanced binary cache in september to see what is still missing—hopefully only edge case things related to my specific configuration.
---
### Author:  [@dramforever](https://discourse.nixos.org/u/dramforever)
#### Post date: [June 28, 2020, 1:28pm UTC](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369/20 "2020-06-28T13:28:35Z")
Ah, `store-paths.xz`.
As I maintain the [TUNA Nix mirror](https://discourse.nixos.org/t/announcing-tuna-nix-mirror/6144) (I maintain the download script, TUNA provides the server), I also need to copy entire channels. I have also used `store-paths.xz` but as you have seen it only includes almost every path, not really all of them. For this reason in the help page
It seems that `store-paths.xz` is generated by Hydra, corresponding to the jobs of a channel, and posted by [this script](https://github.com/NixOS/nixos-channel-scripts/blob/f3aa3bae4c06453fb9012abdbbd9ee422e6d5669/mirror-nixos-branch.pl#L149), whereas [cache.nixos.org](http://cache.nixos.org) consists of every(?) store path Hydra produces.
I haven’t dug much further than that though.
[Next page](https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369.md?page=2)