# Solving the cache.nixos.org problem
**URL:** https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964
**Category:** Development
**Created:** [December 3, 2024, 11:34pm UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964 "2024-12-03T23:34:19Z")
**Posts on this page:** 18
**Page:** 1
### Author:  [@poelzi](https://discourse.nixos.org/u/poelzi)
#### Post date: [December 3, 2024, 11:34pm UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/1 "2024-12-03T23:34:19Z")
Hi,
good news everybody.
There is now a fully decentralized, cheap and fast storage network that we can utilize to create a paid, super fast, 100% uptime cache.
> **[Walrus | Secure Decentralized Storage](https://www.walrus.xyz/)**
>
> Welcome to Walrus, the next generation of data storage. Store media, AI datasets, websites, and more with lightning-fast reads and writes.
It utilize the power of [sui](https://sui.io/), to coordinates storage nodes that store red-stuff encoded chunks of data.
Sui, is more a programmable object database with functions to implement behavior that runs on a worldwide DAG. It basically scales to infinite TPS, depending on the data structures used and has a finality of ~400ms. And cheap, tx cost usually around 0.2$ and can be paid by a third party.
In Walrus, you store data over epochs (1 day). For example, you want to 500 gb for 30 days, then you pay a certain amount of WAL tokens into the storage objects, and the system ensures the data is available and missing chunks restored.
Those who run a walrus node, earn WAL tokens for providing storage.
The data is highly distributed, but can be reassembled very CPU efficient Only 1/3+1 chunk need to arrive for reconstruction.
Data can be stored as read-only or encrypted.
Write is expensive, storage is cheap, read is for free.
1. Define the storage objects we need and how the blobs and signatures should be stored.
2. Create a SUI smart contract that accept any coins, uses a aggregator to convert them into WAL tokens and pays the archive.
3. Create a fast rust daemon that can serve the archive.
- Add local and s3 caches interface
4. Create tooling, allow to run custom archives, …
Walrus is currently in testnet
> **[Walrus Explorer - Walruscan](https://walruscan.com/testnet/home)**
>
> Walrus
[https://tusky.io/](https://tusky.io/) is a dropbox based on walrus and many more are comming.
You can test it out, and get some test sui / test walrus token.
I think this would solve the [cache.nixos.org](http://cache.nixos.org) problematic perfectly.
You can run your own gateway in your local network and get a fully saturated link.
NixOS foundation can then work with a CDNs to serve [cache.nixos.org](http://cache.nixos.org) distributed
---
### Author:  [@Atemu](https://discourse.nixos.org/u/Atemu)
#### Post date: [December 4, 2024, 3:32am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/2 "2024-12-04T03:32:21Z")
> [@poelzi](#):
>
> WAL tokens
Read until there.
Your experimental decentralised data storage (read: extremely complex to get anywhere near usable) that is based around some shitcoin that is very likely just a grift to make someone rich is not a practical solution.
---
### Author:  [@rjpc](https://discourse.nixos.org/u/rjpc)
#### Post date: [December 4, 2024, 3:42am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/3 "2024-12-04T03:42:54Z")
[ ](https://www.youtube.com/watch?v=I2AAUzrse5A)
---
### Author:  [@ksvivek](https://discourse.nixos.org/u/ksvivek)
#### Post date: [December 4, 2024, 8:41am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/4 "2024-12-04T08:41:21Z")
Also check out Arweave.
Odysee is migrating their storage to them. Not sure how much it’ll help
---
### Author:  [@FliegendeWurst](https://discourse.nixos.org/u/FliegendeWurst)
#### Post date: [December 4, 2024, 8:44am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/5 "2024-12-04T08:44:01Z")
Seems like a similar “token” system:
> The protocol is secured by an open and decentralized network of [**miners**](https://arwiki.wiki/#/en/arweave-mining), who store and replicate data in exchange for AR token rewards.
> **[Arweave Wiki](https://2hsfyi4t5fiqdcanybdez4e4admrjeqghts22viz7uuo3d5k2nna.arweave.net/0eRcI5PpUQGIDcBGTPCcANkUkgY85a1VGf0o7Y-q01o/#/en/main)**
>
> A decentralized wiki for Arweave.
---
### Author:  [@poelzi](https://discourse.nixos.org/u/poelzi)
#### Post date: [December 4, 2024, 9:48am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/6 "2024-12-04T09:48:08Z")
Arwave does not scale and is slow. You pay for 100 or 200 years upfront.
Sia and Filecoin are both slow. Here a even longer [list](https://www.techtarget.com/searchstorage/tip/Comparing-4-decentralized-data-storage-offerings)
With Sui and Walrus, we can actually implement cache dropping of packages that have not been requested of a certain amount of time.
Even cooler, we can decentralize the build as well.
We could utilize [ika](https://ika.xyz) which will go live in the next months to sign packages.
Build Queue is in Sui. People can run their a builder, that gets their jobs from the queue.
Each job is distributed to set of different builders, each builds and signes the output hash of the build and uploads the chunks to walrus.
Once enough identical results are uploaded, the result is signed with a key that resides in the MPC network.
This way, the signing key is distributed among hundreds of entities and can only be used through a contract.
We can build faster through better parallelization.
---
### Author:  [@poelzi](https://discourse.nixos.org/u/poelzi)
#### Post date: [December 4, 2024, 10:20am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/7 "2024-12-04T10:20:05Z")
> [@Atemu](#):
>
> Read until there.
>
> Your experimental decentralised data storage (read: extremely complex to get anywhere near usable) that is based around some shitcoin that is very likely just a grift to make someone rich is not a practical solution.
First, it is not mine.
Shitcoin, [sure](https://docs.sui.io/paper/tokenomics.pdf).
I find it fascinating how the whole class of byzantine fault tolerant systems is somehow the evil. I suspect this overreaction comes mostly from people that ignored BTC multiple times and are now angry to have missed out.
That walrus has a token, is the only logical mechanic. You need stake to prevent attacks, and you want participants to earn for providing storage. The WAL token will represent the value of storage of data on a global scale. This way you decouple the price from unrelated values which is a good thing in my opinion.
Not all tokens/coins are a scam - the largest Ponzi schema is inflationary money. Have you never wondered why your parents could build a house and you can’t ?
If you don’t believe in the value of a coin, you can send USDC or other stable coins in 0.5s worldwide for sub cents.
If you prefer to feed the cooperate overlords that constantly attack and harass people for no good reason like paypal,…
---
### Author:  [@poelzi](https://discourse.nixos.org/u/poelzi)
#### Post date: [December 4, 2024, 10:26am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/8 "2024-12-04T10:26:02Z")
If you want to understand what they are building and why, very good interview
[ ](https://www.youtube.com/watch?v=-BUP4da0noY)
There are reasons they win research prices nearly every month and ship…
---
### Author:  [@ksvivek](https://discourse.nixos.org/u/ksvivek)
#### Post date: [December 4, 2024, 11:21am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/9 "2024-12-04T11:21:01Z")
I feel arweave is mostly for permanent (long term) storage and walrus is for faster large size files in a blockchain.
What a year to atleast know all these existing 🔥, Walrus, Tigerbeetle, Arweave, Nix …
I hope 2025 helps me explore more 🔥
---
### Author:  [@poelzi](https://discourse.nixos.org/u/poelzi)
#### Post date: [December 4, 2024, 12:54pm UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/10 "2024-12-04T12:54:45Z")
> [@ksvivek](#):
>
> I feel arweave is mostly for permanent (long term) storage and walrus is for faster large size files in a blockchain.
Yes, but you can build a permanent storage via a smart contract that feeds the storage objects, which is a much better design.
You could for example stake, collect the rewards and pay with the rewards the storage pool.
Walrus will open source once mainnet launches, which is planned for ~ January. I guess the rust lib will then also be available.
---
### Author:  [@srd424](https://discourse.nixos.org/u/srd424)
#### Post date: [December 4, 2024, 2:14pm UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/11 "2024-12-04T14:14:23Z")
I’m normally beyond skeptical of ‘crypto’, but a system for rewarding owners for their machines doing actual useful work, e.g. storage, doesn’t seem necessarily inherently cursed … the devil would definitely be in the technical detail though, I don’t have the brain to evaluate that.
If nothing else, given the likely engineering resource required, you’d want to be reasonably sure any given platform had a long-term future.
Having said that, proof of concepts are always good, if anyone wants to get stuck in?
---
### Author:  [@malikwirin](https://discourse.nixos.org/u/malikwirin)
#### Post date: [December 4, 2024, 4:26pm UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/12 "2024-12-04T16:26:50Z")
I can understand doubt about the crypto community. But tokenization is the only way to monetarily incentivize a system without loosing decentralization.
---
### Author:  [@poelzi](https://discourse.nixos.org/u/poelzi)
#### Post date: [December 4, 2024, 11:01pm UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/13 "2024-12-04T23:01:23Z")
In fact, I think tokenization of communities can create a very strong binding character.
Since the community controls the token mechanics, it can incentivize participation.
Answering forum questions, committing code, developing code, compiling packages, etc could all be rewarded with the community token.
Then you vote about development, and you burn tokens for voting power. Or you stake your token to earn voting power and the longer you stake, the higher your voting power gets. Something like MIN(token_years, token_4) - this way being invested long term gives you more power.
Move is actually a pretty nice language to develop in. Most APIs I have seen in blockchains are like eating glass.
> **[The Move Book - The Move Book](https://move-book.com/)**
>
> First book about the Move programming language and the Move VM. Move documentation, Move tutorials and language reference.
We will need to wait until walrus gets open sourced since it contains the rust libs. They say January.
> [@srd424](#):
>
> I’m normally beyond skeptical of ‘crypto’, but a system for rewarding owners for their machines doing actual useful work, e.g. storage, doesn’t seem necessarily inherently cursed … the devil would definitely be in the technical detail though, I don’t have the brain to evaluate that.
I love crypto, but I’m not a fan of PoW L1 chains like bitcoin. It has advantages, but the waste of energy because the algorithms are so simple is just sad.
I know people that mine bitcoin ethically, like living autonomous and when the batteries are full, they turn on the miner. Or using mining to stabilize the grid in overproduction cases.
I prefer PoS networks, since they are efficient and binding capital as security and inflationary element makes sense. Liquid staking token is your friend…
From what I can see, MystenLabs really tries to build the best distributed development platform on the planet, and their speed of development and research is really impressive. [protocol updates](https://github.com/MystenLabs/sui/commits/main/crates/sui-protocol-config/src/lib.rs) for example.
It will also become more distributed in the next months and DOS resistant (SCION).
---
### Author:  [@VTimofeenko](https://discourse.nixos.org/u/VTimofeenko)
#### Post date: [December 5, 2024, 12:19am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/14 "2024-12-05T00:19:22Z")
> [@poelzi](#):
>
> In fact, I think tokenization of communities can create a very strong binding character.
> Since the community controls the token mechanics, it can incentivize participation.
Or it can drive away anyone who has not been exposed to the cryptocurrency ecosystem, has no idea what any of the first-class concepts mean and has little desire to find it out because it looks daunting.
As someone who has been a part of technology selection processes on buying and selling side (completely unrelated to NixOS), I don’t think you picked the right pitching technique.
When one wants to make a choice between technologies, it’s important to focus on the question “what problem are we trying to solve” and (depending on the complexity and comparability of the solutions), produce some metrics that would make the comparison between solutions easy.
It’s also important to add context to technical details when surfacing them to the top level of the comparison. Few people care about the material from which the bearings are made, people do care that the thing has wheels.
Qualitative notions (cheap, expensive) don’t matter without context. An industrial lathe is probably expensive if one is buying it for their hobby use but cheap if it’s just another addition to a machine shop at a factory.
To illustrate, quoting the original post:
You introduce the concepts of the “network”, “sui”(as a programmable(why does it matter in this context?) object database) and “cheap”(unquantified except for a reference to “transaction” but without defining “transaction”) and “walrus”. Is walrus a business entity? Is it a technology? A combination?
> Paid
Who pays whom? For what specific action?
> super fast
How fast is fast? What specifically is fast? I’m not being facetious, but if we think about which actors are involved in a package cache use, we have:
1. Entity(1) that downloads and builds packages
2. Entity(1) uploads packages to durable storage (2)
3. Packages from storage get distributed to users (3) which could be humans or machines
The “users(3)” is a very heterogenous set (e.g. geography). Would everyone get very download speeds regardless of their location? Would users in the physical vicinity of their nearest “gateway” get good download speeds? How is this achieved?
> Write is expensive, storage is cheap, read is for free.
Again, without qualifying those the numbers do not really make sense.
> 1. Define the storage objects we need and how the blobs and signatures should be stored.
The “define” part, it’s probably “I need 10T of data accessible at Y speed”, but do the definition knobs actually expose what is needed for [cache.nixos.org](http://cache.nixos.org)?
> …how the blobs and signatures should be stored.
How _should_ they be stored? What are blobs in this context? What is signed? Who is validating the signatures? Mentioning concepts like this is again exposing implementation details that produces more questions than answers.
> 1. Create a SUI smart contract that accept any coins, uses a aggregator to convert them into WAL tokens and pays the archive.
Smart contract? WAL tokens? “Pays the archive”? What is the archive?
> 1. Create a fast rust daemon that can serve the archive.
> Add local and s3 caches interface
This sounds very much like “draw the rest of the owl”.
> - Create tooling, allow to run custom archives, …
And this sounds like “draw some friends for the aforementioned owl”.
* * *
I’m in favor of being open minded and giving ideas fair shots, but IMO this is not the way to pitch a technology.
---
### Author:  [@Nebucatnetzer](https://discourse.nixos.org/u/Nebucatnetzer)
#### Post date: [December 5, 2024, 7:54am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/15 "2024-12-05T07:54:44Z")
Things like Seti or Tor relays have existed long before tokens and the only motivation needed for people is to be part of something bigger and maybe the ranking.
In addition attaching voting power to it, is very undemocratic.
---
### Author:  [@zimward](https://discourse.nixos.org/u/zimward)
#### Post date: [December 5, 2024, 11:47am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/16 "2024-12-05T11:47:14Z")
While it is true that those projects have existed for a long time without monetary incentives does not mean that they are sustainable and can lead to adverse effects like secret services running a significant portion of the TOR network. In our case it is questionable if the required bandwith and storage would be available in all regions, something that a monetary incentive would at least alleviate.
I think for an incentive free decentralized approach the barrier to entry (cost, setup complexity) has to be very low to succeed, like peer to peer serving of the store content of nixos machines.
That being said having hardware belonging to the foundation in a co-location datacenter might be just as cost effective with less technological overhead.
---
### Author:  [@waffle8946](https://discourse.nixos.org/u/waffle8946)
#### Post date: [December 12, 2024, 3:42am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/17 "2024-12-12T03:42:53Z")
Aside from critiquing the jargon-y presentation of the original post (like most crypto schemes seem to use), the main aspect that bears stating is, like any cryptocurrency, its inherent value is 0; all value comes from speculation. Using stablecoin as a defense here doesn’t work, because “stablecoin” [isn’t even stable](https://cointelegraph.com/news/usdc-depegs-as-circle-confirms-3-3b-stuck-with-silicon-valley-bank), either. And of course, crypto exchanges have control to misuse the funds stored in said wallets, by say, [coercively converting all funds into a currency that they have full control over](https://x.com/Momentum_6/status/1572616720162127872). Throwing “smart contracts” (which is a fancy way of saying “ad-hoc code that mediates your funds”) into the mix will only serve to [open up a new class of vulnerabilities](https://cointelegraph.com/news/thala-recovers-25-million-exploiter-hacker-caught) with potentially unfortunate legal, financial and technical implications ([which can include things like transferring or deleting all your funds](https://x.com/_TomHoward/status/1567540424608460800)).
At least, with traditional currencies, if a bank cannot pay you back, you have some assurance from the federal governments of most countries where this cache would operate; with crypto, losses are simply your own problem, and you can be stolen from via malicious code, code vulnerabilities, by the exchanges or by the same regulatory features that control most any financial transaction. I think it’s disingenuous to claim that crypto is decentralised in most countries in 2024, nor does it appear to improve the financial position of the NixOS Foundation to throw tens of thousands of euros into yet another such scheme backed by closed-source code.
In any case, it seems like it’s forgotten that there _already is_ a (nearly 5-year old, now) mirror of the nixpkgs cache with (afaik) zero monetary incentive: [Announcing TUNA Nix mirror](https://discourse.nixos.org/t/announcing-tuna-nix-mirror/6144)
Sustainability is good to think of; regardless it seems NixOS has a lot to learn from other distros/projects in terms of distribution methods, rather than creating new problems for ourselves with novel, unproven methods.
---
### Author:  [@tennox](https://discourse.nixos.org/u/tennox)
#### Post date: [December 16, 2024, 10:44am UTC](https://discourse.nixos.org/t/solving-the-cache-nixos-org-problem/56964/18 "2024-12-16T10:44:13Z")
Thank you @poelzi for your suggestion - I think it’s worth to think about decentralized storage (and the nix community [already did](https://discourse.nixos.org/t/why-does-the-nixos-infrastructure-have-to-be-hosted-in-a-centralized-way/46789)) - but please stay humble and realistic.
Suggesting a technology that hasn’t even reached mainnet for a large-scale project that needs stable infrastructure - I think we can talk about that in 1 year - _earliest_… When some other, more experimental projects have adopted the technology, and the network has proven itself in terms of stability and longevity.