Vision for nixpkgs

Development
1

Since nixpkgs is constantly moving, i think a vision describes it best.

I want to propose a vision that is specific enough, so we can work towards a common goal, but broad enough, so everyone approves it.

I want to add this text to the README and the documentation.

The vision we work towards

  • extensive collection of relevant free software packages
  • that are built from source code (and available from our binary cache)
  • are up to date (assisted by automation)
  • and work as expected (ensured by automatic tests)
  • deliver upstream’s original intent (WIP: wording will get improved)
  • with support for linux and macOS (and potentially other platforms like BSD and Windows)
  • unfree software packages are allowed for those who need them

I also propose this as short and precise description:

nixpkgs - extensive collection of free software packages for linux and macOS

This follows the theme that we don’t want to promote unfree software, but also don’t hide it from people who search for it.

What do you think? Any suggestion to improve it?

cc @garbas @edolstra

21 Likes
2

Nice. Another thing I feel is part of the nixpkgs vision is trying to deliver upstream’s original intent. We try to avoid patching, themeing, etc.

7 Likes
3

From a marketing perspective, we probably shouldn’t mention “built from source code” without also mentioning our amazing cache.

https://twitter.com/GabrielG439/status/1361744122042667010

8 Likes
4

Thanks for this.

It’s really had me thinking; should nixpkgs be more stringent about excluding artifacts that do not build from release?

While it’s much easier to make that argument for machine-code; I’ve come across a number of derivations for VMs such as Java.

You’ve at least inspired me to fix JRuby’s derivation.

It simply downloads the compiled Java JAR (artifact) rather than building from source.
The ability to easily change to an arbitrary commit and get a different package is one of the fundamental superpowers Nix gives.

I’ve been working on the improvement here. I hope to polish it this week for submission.

8 Likes
5

I am in support of a passthru or metadata flag to easily identify* whether a derivation is built from true source or not.

Again, this will discount many VM artifacts like: Ruby Gems, Python Eggs, Java JAR’s etc…

4 Likes
6

As said, i would frame it as vision that everything is built from source, but reality is that it’s sometimes hard. In that case i value to have a (binary) package over no package.

Same with tests. Right now very few packages have tests. It’s not even documented how to create a package test. (i want to change that)

To work on such goals it would be helpful to get a % how many packages are built from source/have tests/… There metadata can help.

8 Likes
7

You might also want to mention stability through channels :

  • nixos-xx.xx : stable but not bleeding edge and major update every 6 month
  • unstable : more bleeding edge but may be somewhat unstable

I think it is important to show that nixpkgs supports different use cases. For example servers will probably stick to stable releases while power users will probably want the most up to date system even if it means living with a bit of unstability.

I’m not sure how I would word it though without undermining the “up to date” and “work as expected” too much.

@davidak I really like the initiative btw!

4 Likes
8

I really like the idea to have some precise stat about how much package are build from source or doesn’t depend on precompiled software outside of bootstrap (since I have read all the guix manual a year ago). Will pick some random package, and take a look at how many of them are build from source.

2 Likes
9

There is an RFC for that: [RFC 0089] Collect non-source package meta attribute by risicle · Pull Request #89 · NixOS/rfcs · GitHub

10

most of the vision is a reality nowadays

I would love to see in a place so important like the README.md
something that tells “Hey, look, this is good for you”.
so it would be awesome to mention facts as:

  • statistically the most extensive (package number) repository out there
  • statistically the most updated repository out there, which makes it also very secure because new CVEs are discovered every day and usually latest versions are the only ones that are free of known vulnerabilities
  • the most stable software repository (hashes everywhere), just pin to a commit and enjoy
  • secure against supply chain attacks (hashes everywhere), pin and enjoy
  • everything from source and official sources

Those features are not easy to see at a first glance,
but they exist and add a lot of value

Would be nice to expose them

2 Likes
11

That’s great. In fact, I would advocate to remove all the statements that don’t closely match reality. That way a visitor can read that section and quickly be up to speed on the overall spirit of the project.

5 Likes
12

That’s not what a vision is. It is the project status. We had that and it was always outdated. That’s why i suggested this format.

It is important to me to use the words that say precisely what i mean, so i looked up what vision mean (in this context) and learned a few things how to write a good vision statement.

A vision statement must be written in infinite terms and is not changeable, once decided it is locked. If the company changes the vision based on events like market conditions, current technology, or social media trends the company risks the very foundation for its creation. If it changes its vision on whims, it will inevitably perform poorly financially, operationally, technologically, or all of these because it is making decisions based on near term results only.

Source: Vision statement - Wikipedia

Maybe a vision statement is too big for nixpkgs, because it’s a product in itself.

Products and services that one wants to provide are never mentioned in a vision. If it did, the company vision would be limited only to producing what was mentioned in the vision. (Not only that, it’s boring and uninspiring to do so.)

So i think we need a vision statement for the whole NixOS project.

it will inevitably perform poorly financially, operationally, technologically

i think this is kind of the case with the project. compare the success of docker and our success. (of course docker is a for-profit effort with heavy marketing, but it also solves a problem developers have. i remember how everyone was talking how great it is)

i think the current state of the project is that eelco had a great idea 18 years ago and we are only adding onto it. there is no big vision or goal we work towards. every contributor has their own goals and the whole project is a complete mess and has a bad user experience. the current users are enthusiasts that see the potential of the idea, but we are far away from becoming mainstream. i think a vision statement can help cooperate better and provide a better solution for our users (and us)

but this post is becoming a rant again. maybe these thoughts are just a product of my personal needs. maybe i want to have some kind of control, bring this chaotic project in order. maybe that’s exactly what it needs for success? i am going through some existential challenges right now. stop me if you think it does not benefit the project

tl;dr a “vision statement” is too big for nixpkgs. i will just call it goals and think about a vision statement for the whole nixos project

8 Likes
13

Vision:
“Fundamentally change the way software & systems are built and deployed to provide reproducibility.”

5 Likes
14

statistically the most updated repository out there, which makes it also very secure because new CVEs are discovered every day and usually latest versions are the only ones that are free of known vulnerabilities

This might be true for unstable, but this argument doesn’t apply for stable releases;
Worse, the argument “most recent being equivalent to with fewest bugs” is a fallacy - new code often introduce bugs!
And while we have “Vulnerability Roundups” (shoutout to all people fixing the issues and working on the tooling), we are far from being a match for the distributions with dedicated security engineering teams.

2 Likes
15

Yes, it replaces old bugs with new ones. You can pick your poison. :slight_smile:

16

I think we should have a vision, and I hope @davidak changes his mind back and adds one to the Nixpkgs readme.

8 Likes
17

I think it benefits the project and I would love to see what you propose happening

From my experience in order to create curiosity on new users it’s important to let them know in the front page:

Hey, look, this is good for you, it solves many problems!

“Vision”, “Why”, “Features”, “Goal”, those are awesome sections for this and we currently don’t have any

I think we can start with something, and then iterate!

so IMHO: let’s do it! it is an improvement

3 Likes
18

I also miss the reproducibility aspect in the vision list, but then I fail to come up with a good slogan, because it seems mostly refer to aspects of nix itself (reproducible build steps), or NixOS (reproducible setups).

But since visions can also list things that we don’t have yet, but want to have eventually, how about a striving for reproducibility in the sense of https://reproducible-builds.org/? That is clearly a vision for nixpkgs (rather than more of nix or NixOS).

6 Likes
19

I don’t think bitwise is a hard goal nixpkgs (Nix) is striving for.
To me some of the best win & vision statements from Eelco’s thesis is the pragmatism that Nix has.

It would be cool if that pragmatism is shown in the vision & goal.

pragmatism: source level reproducibility, using grep to find runtime dependencies, patchelf etc…

4 Likes
20

:+1: for mention of pragmatism!

1 Like