How to update VSCode Insiders?

Help
1

I’m using this configuration from official wiki:

(pkgs.vscode.override { isInsiders = true; }).overrideAttrs (oldAttrs: rec {
  src = (builtins.fetchTarball {
    url = "https://code.visualstudio.com/sha/download?build=insider&os=linux-x64";
    sha256 = "";
  });
  version = "latest";

  buildInputs = oldAttrs.buildInputs ++ [ pkgs.krb5 ];
});

And now I can’t update the installation. I removed the hash, assuming it will re-download the package to check the hash, but it just reads hash from the old tarball, which resides somewhere in cache.

So how to make nix re-download the tarball?

2

Does the same also happen if you use sha256 = lib.fakeSha256?

3

yes. The problem is that the tarball is somewhere in /nix/store, and the builder just check the hash against that previously downloaded version. Lazy shit :unamused:

4

Bold move to accuse others of laziness when you’re asking other people to help solve your problem for you.

5

Are you… sure? Because nix normally stores the files with the hash making up part of their path, if you change the hash the path should change, and nix should realize that the path does not yet exist. Perhaps you’re getting a 404 page or something, and that just repeatedly gives you the same hash.

It should be noted that the wiki is both unofficial and notoriously out of date, and in places completely wrong. It wouldn’t surprise me if the link had changed since someone last touched that page.

Either way, a nix-collect-garbage -d should remove any store paths not currently used by anything, if you still have an issue after that, please share the exact error, and check the contents of what nix actually downloads.

Pretty sure it’s supposed to be a joke on nix’ laziness, y’know, lazy evaluation. In this case the issue definitely isn’t that, to be fair, it’s either a massive bug in nix’ data storage or user error.

1 Like
6

Ah, if that’s the case then please accept my apologies.

fetchTarball has a caching layer above the Nix store, and it both uses a TTL (one hour by default) and checks the server’s ETag before downloading a URL a second time. If Microsoft isn’t correctly updating the ETag published by the server, that could be causing this problem.

The tarball cache is stored in ~/.cache/nix/fetcher-cache-v1.sqlite, and if you want to, say, delete the last day of cached entries, you can run

sqlite3 ~/.cache/nix/fetcher-cache-v1.sqlite "delete from Cache where timestamp > $(date +%s) - 86400;"
1 Like
7

I’m accusing my own software, but maybe you’re right, since AI is getting more powerful I’d better change this habit :yum:

2 Likes
8

No, ~/.cache/nix not so big to contain vscode’s binary:

du -sh .cache/nix
764K    .cache/nix

but home-manager switch still gives an error:

error: hash mismatch in file downloaded from 'https://code.visualstudio.com/sha/download?build=insider&os=linux-x64':
         specified: sha256:0000000000000000000000000000000000000000000000000000
         got:       sha256:1gkyagbrkihxbzcw8z0rg7plvgp7gg7yq6nlc0r7hwczsgz1dj1j

And somehow it gets the hash of the installed version (I checked with the previous version of the config). So apparently the old downloaded binary is stored somewhere in /nix/store.

9

The SQLite cache just stores the URL(s), the hash, the ETag (if there was one), a timestamp, and the path in the Nix store containing the actual data. If you try to fetch the URL, and the server responds with an ETag matching the one in the database, then fetchTarball acts as if it just completed a download and the result is the Nix store path indicated.

10

I extracted relevant data from fetchTarball’s cache at ~/.cache/nix:

{"name":"source","type":"file","url":"https://code.visualstudio.com/sha/download?build=insider&os=linux-x64"}|{"etag":"\"0x8DBBB578A1BBD29\"","url":"https://az764295.vo.msecnd.net/insider/c72447e8d8aaa7497c9a4bd68bc4301584b92beb/code-insider-x64-1695374990.tar.gz"}|/nix/store/yh391bg0s3d7z15g67012ggvvqgds004-source|1|1695525867
{"name":"source","type":"tarball","url":"https://code.visualstudio.com/sha/download?build=insider&os=linux-x64"}|{"etag":"\"0x8DBBB578A1BBD29\"","lastModified":1695374990}|/nix/store/6djz6i4yzr774sx65x6wmjyas2z3c9c8-source|1|1695525870

Can you say please what’s the problem here?

11

Having extracted those store paths, run

nix-store --delete /nix/store/yh391bg0s3d7z15g67012ggvvqgds004-source /nix/store/6djz6i4yzr774sx65x6wmjyas2z3c9c8-source

and try your update again.

12

Thanks!

One got deleted, but another one have gc-roots:

❯ nix-store --delete /nix/store/yh391bg0s3d7z15g67012ggvvqgds004-source

finding garbage collector roots...
deleting '/nix/store/yh391bg0s3d7z15g67012ggvvqgds004-source'
deleting unused links...
note: currently hard linking saves 12755.37 MiB
1 store paths deleted, 0.00 MiB freed
❯ nix-store --delete /nix/store/6djz6i4yzr774sx65x6wmjyas2z3c9c8-source

finding garbage collector roots...
0 store paths deleted, 0.00 MiB freed
error: Cannot delete path '/nix/store/6djz6i4yzr774sx65x6wmjyas2z3c9c8-source' since it is still alive. To find out why, use: nix-store --query --roots
❯ nix-store --query --roots /nix/store/6djz6i4yzr774sx65x6wmjyas2z3c9c8-source

/home/joe/.local/state/nix/profiles/home-manager-11-link -> /nix/store/h17b1j8mvj4c4la9kfgr428issfza8fm-home-manager-generation
/home/joe/.local/state/home-manager/gcroots/current-home -> /nix/store/h17b1j8mvj4c4la9kfgr428issfza8fm-home-manager-generation
/proc/1242/fd/15 -> /nix/store/jj4yf6d24fhkd2ix4ihmcfw407bcfgwq-home-manager-path
/proc/77833/maps -> /nix/store/jj4yf6d24fhkd2ix4ihmcfw407bcfgwq-home-manager-path
/proc/1588/fd/22 -> /nix/store/jj4yf6d24fhkd2ix4ihmcfw407bcfgwq-home-manager-path
/proc/1588/maps -> /nix/store/jj4yf6d24fhkd2ix4ihmcfw407bcfgwq-home-manager-path
....

Running home-manager switch still gives the same error.

I’m also wondering why those entities were still in ~/.cache/nix/... from Sept’23?

13

Ah, in that case the safest thing would probably be to delete the entry from the cache.

My oldest cache entry is from June 2022; pretty sure they just hang around in there until they’re invalidated. If servers are truthful about when their resources change, it shouldn’t matter.

14

I checked ETag just now, it’s different from those in the cache.

So the problem is with fetchTarball: when wrong hash provided it checks it against expired ~/.cache/nix entities (2 months vs 1 hour TTL) without even checking new etag.

curl -I "https://code.visualstudio.com/sha/download?build=insider&os=linux-x64"
HTTP/2 302 
date: Tue, 19 Dec 2023 10:03:45 GMT
content-type: text/plain; charset=utf-8
content-length: 165
location: https://vscode.download.prss.microsoft.com/dbazure/download/insider/a017b12b9caa3475675cfe6fda014fcf5af388c9/code-insider-x64-1702964236.tar.gz
vary: Accept
strict-transport-security: max-age=31536000; includeSubDomains
content-security-policy: frame-ancestors 'self'
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
x-powered-by: ASP.NET
x-azure-ref: 20231219T100344Z-x5x36pe0yx0018809h5c1cs7kc00000003900000000004sy
x-cache: CONFIG_NOCACHE
set-cookie: ASLBSA=0003fd61d1353cd3703adafaf5302ae7df61c93c551d2a077c0ece56ca05e1a8ee10; Path=/; Secure; HttpOnly;
set-cookie: ASLBSACORS=0003fd61d1353cd3703adafaf5302ae7df61c93c551d2a077c0ece56ca05e1a8ee10; SameSite=none; Path=/; Secure; HttpOnly;

❯ curl -I "https://vscode.download.prss.microsoft.com/dbazure/download/insider/a017b12b9caa3475675cfe6fda014fcf5af388c9/code-insider-x64-1702964236.tar.gz"
HTTP/2 200 
accept-ranges: bytes
age: 10933
cache-control: public, max-age=86400
content-disposition: attachment; filename=code-insider-x64-1702964236.tar.gz; filename*=UTF-8''code-insider-x64-1702964236.tar.gz
content-type: application/octet-stream
date: Tue, 19 Dec 2023 10:05:47 GMT
etag: "0xCC7F2B5ACABD5CD91E6643FFCA646B27F0C3213528158908672B9B9DD663D915"
last-modified: Tue, 19 Dec 2023 06:24:24 GMT
server: ECAcc (hkc/BDBF)
x-cache: HIT
x-ms-apiversion: Distribute 1.2
x-ms-region: prod-weu-z1
content-length: 139998654
15

Hmm, the plot thickens.

What do you get if you run

nix-build --debug --expr 'builtins.fetchTarball {                                                        
  url = "https://code.visualstudio.com/sha/download?build=insider&os=linux-x64";
  sha256 = "";
}'
16

I removed those entries from the ~/.nix/cache/fetcher-cache-v1.sqlite, and all worked as charm: it re-downloaded the tarball, gave me new hash, I put it into config and updated vscode insiders. Thanks everybody! :partying_face:

But I doubt nix users supposed to do it manually every time, ah? Maybe it’s just some once-only fault with my lazy fetchTarball?

17

Hard to say, but you may have encountered a legit bug. I don’t think impure URLs (returning different contents at different points in time) are used very much with Nix, so it wouldn’t surprise me if there are rarely exercised edge cases in the caching code that don’t work as intended.

If this happens again the next time you try to update Insiders, I would:

  • Replace the "https://code.visualstudio.com/sha/download?build=insider&os=linux-x64" URL with whatever it expands to ("https://vscode.download.prss.microsoft.com/dbazure/download/insider/a017b12b9caa3475675cfe6fda014fcf5af388c9/code-insider-x64-1702964236.tar.gz" today, something else tomorrow) to get your setup working.
  • Report the issue at Issues · NixOS/nix · GitHub, because I don’t see a current issue corresponding to this behavior. Try running an isolated nix-build on just the fetchTarball call, as in my last comment to you, making sure to include the --debug flag, and include it in the report; it’ll probably help with figuring out what code paths Nix is using.
1 Like