# Nixpkgs CLI Working Group Member Search
**URL:** https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517
**Category:** Nixpkgs Architecture
**Created:** [July 16, 2023, 4:00pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517 "2023-07-16T16:00:45Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@Growpotkin](https://discourse.nixos.org/u/Growpotkin)
#### Post date: [July 16, 2023, 4:00pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/1 "2023-07-16T16:00:45Z")
# Nixpkgs CLI working group
The Nixpkgs Architecture Team wants to convene a working group for coming up with a CLI tool for Nixpkgs.
This tool would have a similar relationship to Nixpkgs as `nixos-option` does to NixOS.
Like `nixos-option` it should live in Nixpkgs so it can be kept in strict sync.
We’ll be organizing this working group with regular hacking sessions, so if you’re interested in joining - let us know! Our first task will be creating a tool to efficiently scrape package metadata from `nixpkgs` ( more info below ).
We’re looking for a shepherd for this project and additional contributors to get the ball rolling.
## Problem
Nixpkgs has tons of domain-specific concepts that are not appropriate to include in Nix proper.
Currently, we have an awkward compromise where commands like `nix-shell`, `nix develop`, and `nix shell` assume and [“ad hoc, informally-specified”](https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule) subset of these concepts, but this is the worst of both worlds:
- Nixpkgs cannot evolve freely, because it can’t break Nix’s assumptions.Examples:
- [stdenv-setup and misc pkgs: Revert to space-deliminated propagated-\* … · NixOS/nixpkgs@f6f40e3 · GitHub](https://github.com/NixOS/nixpkgs/commit/f6f40e3fe5fbb9721624a218faea1b520f9ec200) could not switch away from space-separation for `propagated-user-env-packages`
- [check-meta: use system tuple in platforms · NixOS/nixpkgs@a52e317 · GitHub](https://github.com/nixos/nixpkgs/commit/a52e317200eca8ef7b8c6d6b77336a6a9f397a0e) / [meta.platforms is broken · Issue #60345 · NixOS/nixpkgs · GitHub](https://github.com/NixOS/nixpkgs/issues/60345) could not switch from list of support platform to more abstract “open world” descriptions
- Nix never does everything people want with these commands._Endless_ issues are opened about them.Examples:
- [https://github.com/NixOS/nix/pull/4702](https://github.com/NixOS/nix/pull/4702) `MANPATH` not being set caues problems. Offloading responsibility to Nixpkgs (as discussed here!) was proposed.
- [nix-env: Use pname and version attrs in -q by jtojnar · Pull Request #4463 · NixOS/nix · GitHub](https://github.com/NixOS/nix/pull/4463) `nix-env --query --meta` has broken hack for of `pname` and `version`.
## Possibility
The benefit aren’t just defensive — avoiding the use of Nix commands which constrain Nixpkgs.
Having separate commands also opens the door to making tons of domain-specific functionality more accessible!
- Far richer meta fields, e.g. with language-specific ecosystem extensions are possible. Can present this information to users much better.
- Contributer tools, like that for [https://github.com/NixOS/rfcs/pull/140](https://github.com/NixOS/rfcs/pull/140), can take advantage of how the code is structured.
- Smart shells that now how to combine packages (for working on multiple things at once) and other tricks are possible.
## Tentative first objective
The “possibility” part is the most fun an exciting.
But it is also open-ended, and that is dangerous for a new working group.
A precise narrow goal can keep things on track, and then momentum can be built for later.
### Task
Replace `nix-env --query --available --meta` invocation in `pkgs/top-level/make-tarball.nix` with invocation of a new CLI command.
### Success criterion
[nix-env: Use pname and version attrs in -q by jtojnar · Pull Request #4463 · NixOS/nix · GitHub](https://github.com/NixOS/nix/pull/4463) is no longer needed because the new command handles `pname` and `version` correctly without needing Nix to understand them.
### Method
Make new CLI residing in Nixpkgs like `nixos-option`.
Invoke it instead.
### Non-goals
#### It doesn’t need to be stable
At first, this is just an internal tool for `pkgs/top-level/make-tarball.nix`.
It can be as unstable as we like, since it and its invocation live in the same repo (Nixpkgs).
Tools for humans can come later, and only then do we need to start thinking about stability.
#### It doesn’t need to avoid unstable internals of Nix
It doesn’t _need_ to use Nix lib as a library, but doing so is explicitly allowed.
There is work to make a new, more stable C interface for Nix to allow using languages other than C++, but we should not wait for it to be ready.
Rather, our new CLI and `nixos-option` can both serve as additional motivation for it to be finished.
# Co-signers
- @Ericson2314
---
### Author:  [@domenkozar](https://discourse.nixos.org/u/domenkozar)
#### Post date: [July 16, 2023, 4:10pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/2 "2023-07-16T16:10:44Z")
Yes! Please.
Related: [Rename `nixos-rebuild` to `nixos` · Issue #122715 · NixOS/nixpkgs · GitHub](https://github.com/NixOS/nixpkgs/issues/122715)
---
### Author:  [@RaitoBezarius](https://discourse.nixos.org/u/RaitoBezarius)
#### Post date: [July 16, 2023, 4:27pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/3 "2023-07-16T16:27:47Z")
I’d be interested into joining this WG, but I have limited bandwidth for the next month, though, in September, I should be fine.
I have various project for nixpkgs CLI that I am dabbling with, oriented to maintainers, contributors and committers.
---
### Author:  [@blaggacao](https://discourse.nixos.org/u/blaggacao)
#### Post date: [July 16, 2023, 5:08pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/4 "2023-07-16T17:08:03Z")
As author of a [tool living in an analogous problem space](https://github.com/paisano-nix/tui) I’d be interested in joining as well with variable (hopefully: upwards trending) commitment that I would also need to negotiate with other stakeholders.
I have a follow up question/intuition on the initial post above:
It is not mentioning things like `nix-init` and `nixpkgs-review`, potentially because of the aim to keep the scope narrow, but if these use cases aren’t completely off-limit, mentioning them might help some people to make the right associations.
EDIT:
Another follow up: How is the relationship with the Nix Team managed and structured (joint sessions, personnel overlap, team-level commitments, etc)? I imagine that this working group would rather sooner or later depend on a excellent collaboration between the two teams to move towards its mission.
---
### Author:  [@water-sucks](https://discourse.nixos.org/u/water-sucks)
#### Post date: [July 16, 2023, 9:24pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/5 "2023-07-16T21:24:34Z")
While not related to a `nixpkgs` CLI per se, I’ve been toying around with writing a replacement for all of the `nixos-*` tools in Zig such as [this issue](https://github.com/NixOS/nixpkgs/issues/54188) describes. Currently, I have plans to unify the following tools:
- `nixos-rebuild`
- `nixos-container`
- `nixos-enter`
- `nixos-generate-config`
- `nixos-install`
- `nixos-version`
- `nixos-help`
And possibly other projects like `nixos-option` and `nixos-infect`, if possible.
Currently, I have a `nixos-rebuild` replacement somewhat working, except it does not implement the remote build/target options, and that would take some (by that, I mean a lot) of refactoring. While I have limited time to work on this, I would love to be able to make this happen sometime in the next few months and to open up the work once I think it is ready to show.
---
### Author:  [@blaggacao](https://discourse.nixos.org/u/blaggacao)
#### Post date: [July 16, 2023, 9:54pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/6 "2023-07-16T21:54:28Z")
Does Zig have a good answer to dynamic CLI completion? Rust (i.e. `clap`) afaik only recently added some support for it, Go had it for a while already via `cobra`. For extensibility or ingestion of metadata that comes out of the target repo, dynamic CLI completion, eventually aided by a cache, is handy.
---
### Author:  [@water-sucks](https://discourse.nixos.org/u/water-sucks)
#### Post date: [July 16, 2023, 10:08pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/7 "2023-07-16T22:08:43Z")
I do not know of any solution for dynamic CLI completion that I wouldn’t have to implement myself in Zig. However, I’m thinking about using Zig’s build system to generate shell completions from the declarations of commands, so that might come in handy later.
This is still very much a rough idea, though, so it’ll definitely take some work to add features like dynamic CLI completion to it. For right now, I just want to focus on mirroring the vital features of the `nixos-*` commands and exposing them with a better interface than what we currently have right now, and expand it out from there.
---
### Author:  [@blaggacao](https://discourse.nixos.org/u/blaggacao)
#### Post date: [July 16, 2023, 10:17pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/8 "2023-07-16T22:17:58Z")
Sounds awesome. I’d be curious to be able to have a look at the prototype (mainly to understand a familiar programming domain in Zig).
Another element that may be worth considering and/or exploring is a plugin system capability. A plugin system unlocks distributed creativity to tackle new `nixpkgs *` problems, but also would serve as a sort of graduation academy to stabilize the best ideas into `nixos/nixpkgs`.
---
### Author:  [@Growpotkin](https://discourse.nixos.org/u/Growpotkin)
#### Post date: [July 16, 2023, 11:23pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/9 "2023-07-16T23:23:03Z")
Just as a heads up I’d like to avoid any excursions into porting Nix to language X as a prerequisite to making the handful of calls we need to the `nix` CLI, or existing `libnix-util`, `libnix-cmd`, and `libnix-expr` APIs.
The focus here is to turn out a functional tool that’s easy to maintain and contribute to. I love experimenting with new programming languages as much as any nerd, but upfront I’d like to say “this is not a working group to create FFI bindings or attempt to port core `nix` routines to other languages”. This is a working group to turn out an efficient and practical CLI tool that will likely recycle some existing Nix C++ routines, the plugin framework, and/or wrapped calls to the `nix` CLI.
This isn’t to say that I discourage anyone who is interested in ports or rewrites, but I do want to be clear that those activities belong elsewhere.
Let’s hack together a fast and useful tool, and try not to get lost in tangents of high effort ports or cross language linkage. The top priorities are user experience and ease of maintenance. ❤
---
### Author:  [@blaggacao](https://discourse.nixos.org/u/blaggacao)
#### Post date: [July 17, 2023, 12:06am UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/10 "2023-07-17T00:06:23Z")
Not sure if there’s a misunderstanding at play or the statement pertains (partially) to a different context.
From the context I gathered the idea to use Zig is that it can seamlessly import the nix headers & al., while being a saner language to achieve the UX and maintenance goals than X.
In short: avoidance of any FFI requirement, but also avoid CLI wrappers.
But it’s also possible that I misunderstood that aspect.
And there are downsides, too, ofc (popularity, stability, ecosystem maturity). 😄
---
### Author:  [@Growpotkin](https://discourse.nixos.org/u/Growpotkin)
#### Post date: [July 17, 2023, 12:49am UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/11 "2023-07-17T00:49:22Z")
Yeah nothing personal and it’s not disparage one language vs another, I just want to maintain focus on outcomes.
Context is: many projects fizzle out because they try to rewrite existing interfaces in another language or pick a language that is inaccessible to the majority of developers/contributors.
The goal is to make a CLI tool - there’ll be plenty of brainstorming useful features, design decisions to hash out, fun technical problems to tackle, and architecture/infrastructure to develop along the way. I want to focus energy on those efforts.
---
### Author:  [@blaggacao](https://discourse.nixos.org/u/blaggacao)
#### Post date: [July 17, 2023, 12:54am UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/12 "2023-07-17T00:54:37Z")
> [@Growpotkin](#):
>
> pick a language that is inaccessible to the majority of developers/contributors
I think this is definitely a key point and we may need to subsume an ecosystem that goes mainstream with thousands of contributors rather than looking at the status quo assessing this.
Whatever the decision will be, I have no intentions to preempt and am looking forward to this new CLI chapter.
---
### Author:  [@Ericson2314](https://discourse.nixos.org/u/Ericson2314)
#### Post date: [July 17, 2023, 7:16pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/14 "2023-07-17T19:16:29Z")
So [(Towards) stable C bindings for libutil, libexpr by yorickvP · Pull Request #8699 · NixOS/nix · GitHub](https://github.com/NixOS/nix/pull/8699) is meant to go hand-in-hand with Python bindings.
The way I see this WG going down is something like this:
1. Work with C++ today, because we can immediately get started.
2. Emphasis on decoupling layer violations in Nix (like the initial `nix-env --query --available`) before adding new misc functionality.
3. Maintainability should initially be achieved with less code, rather avoiding C++ if people find it off-putting.
4. Creating tools like this and `nixos-rebuild` will _help_ not _hurt_ the case for [(Towards) stable C bindings for libutil, libexpr by yorickvP · Pull Request #8699 · NixOS/nix · GitHub](https://github.com/NixOS/nix/pull/8699); the more florishing tools we have using Nix’s unstable C++ interfaces downstream of Nix itself, the more badly we need a sustainable solution.
5. Once [(Towards) stable C bindings for libutil, libexpr by yorickvP · Pull Request #8699 · NixOS/nix · GitHub](https://github.com/NixOS/nix/pull/8699) and the follow-up Python bindings (which may or may not be in tree) are done, we have the _option_ to rewrite the Nixpkgs CLI from C++ to Python, or something else.
---
### Author:  [@Ericson2314](https://discourse.nixos.org/u/Ericson2314)
#### Post date: [July 17, 2023, 7:20pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/15 "2023-07-17T19:20:51Z")
> [@blaggacao](#):
>
> Another follow up: How is the relationship with the Nix Team managed and structured (joint sessions, personnel overlap, team-level commitments, etc)? I imagine that this working group would rather sooner or later depend on a excellent collaboration between the two teams to move towards its mission.
So as someone on the Nix team who helped plan this out with @Growpotkin, this WG would be welcome to collaborate us but I also look forward to you asserting your independence! Per my [[RFC 0136] A plan to stabilize the new CLI and Flakes incrementally by Ericson2314 · Pull Request #136 · NixOS/rfcs · GitHub](https://github.com/NixOS/rfcs/pull/136#discussion_r1246035453) I believe Nix itself is better suited to nailing the core functionality / advanced plumbing, and external tools are better suited to nailing the learning journey / misc bonus features. I would love if external tools, like this, would _lead_ in that regard, and Nix proper would _follow_, providing the foundation atop which the building can be built.
---
### Author:  [@RaitoBezarius](https://discourse.nixos.org/u/RaitoBezarius)
#### Post date: [July 17, 2023, 7:53pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/16 "2023-07-17T19:53:16Z")
While on the subject of CLI and Nix team, could we have a backport of this? [restoreMountNamespace(): Restore the original root directory by edolstra · Pull Request #8483 · NixOS/nix · GitHub](https://github.com/NixOS/nix/pull/8483)
It is still hurting a lot of users out there. 😉
---
### Author:  [@Ericson2314](https://discourse.nixos.org/u/Ericson2314)
#### Post date: [July 17, 2023, 8:19pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/17 "2023-07-17T20:19:40Z")
Oh my, are we on the Nix Team so AWOL that derailing the thread is the only way to get our attention?! 😮 🙂
---
### Author:  [@Growpotkin](https://discourse.nixos.org/u/Growpotkin)
#### Post date: [July 17, 2023, 9:23pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/18 "2023-07-17T21:23:16Z")
Updated to clarify that this Working Group is being recommended by The Nixpkgs Architecture, and that a shepherd/organizer and additional contributors are what we’re hoping to rally for here.
I’ll be contributing but won’t be leading the charge or going it alone on this one ❤
---
### Author:  [@blaggacao](https://discourse.nixos.org/u/blaggacao)
#### Post date: [December 5, 2023, 5:30pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/19 "2023-12-05T17:30:56Z")
[https://github.com/water-sucks/nixos/pull/3](https://github.com/water-sucks/nixos/pull/3)
And if the dear reader is furthermore interested:
[https://github.com/water-sucks/nixos/pull/4](https://github.com/water-sucks/nixos/pull/4)
---
### Author:  [@blaggacao](https://discourse.nixos.org/u/blaggacao)
#### Post date: [December 6, 2023, 1:21pm UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/20 "2023-12-06T13:21:13Z")
@viperML Wouldn’t you have a ton of input, as well, on this topic?
---
### Author:  [@viperML](https://discourse.nixos.org/u/viperML)
#### Post date: [December 7, 2023, 8:24am UTC](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517/21 "2023-12-07T08:24:17Z")
Thanks for the ping, and I wouldn’t mind being involved in the WG.
However, my experience with CLI’s is limited to [nh](https://github.com/viperML/nh), which just calls nix as a subprocess. It’s written in Rust using clap-derive (which I think is fantastic for CLI’s, see [example here](https://github.com/viperML/nh/blob/375c6cf57de3a839b7937358659bea526da27eae/src/interface.rs)) and clap-complete, which generates completions automatically (but I don’t know if it can generate completion dynamically).
Waiting for the bindings is maybe the best bet for writing a tool that “understands” all the metadata.
[Next page](https://discourse.nixos.org/t/nixpkgs-cli-working-group-member-search/30517.md?page=2)