TL;DR: language interpreters and applications with libraries or plugins respectively, organized as a sub-packageset, using an overlay system, with composable
.withPackages (and co.) for All The Things!
Ideally this will result in less duplicated code, more consistency, and a knowable, reliable, and convenient toolset for dealing with the constructs this system wants to deal with. I feel the current implementations for language package sets and application plugin systems are not like this.
This is something I’ve been working on for a while and tried to get feedback, without much luck I feel.
Now I realized I might as well try here as well, and I can reuse this post as the documentation I’m currently lacking.
It’s not a real RFC yet because the implementation is still in flux, and design changes may happen. However, the basic structure feels pretty stable.
You are all cordially invited to shower upon me your desires and criticisms - do your worst!
- library user: someone trying to write nixpkgs stuff
- frontend user: someone trying to install plugins in an app or packages for an interpreter, for example
- (primary) to provide a consistent interface for things that fit into the application+plugin/interpreter+library analogy.
i.e. a consistent user facing frontend for plugins and language infrastructures.
- (secondary) to provide an abstracting library for implementors of language and plugins backends, which supplies common functionality and glue.
This has the upside of, ideally, hiding some unnecessary complexity from the library user.
These goals could in theory be considered separately, but here they are currently handled as one demonstrator implementation.
- should be composable, where appropriate (yes this is vague)
- escape hatches should be availible so the library user may implement their own “local” overriders of some functionality if desired -
i.e. it is explicitly a goal, that it should not be necessary to duplicate unrelated code - “local” “patches” should remain “local”.
I personally find large, unmodifiable let expressions in nixpkgs rather annoying.
- Provide amenities to make library user life easier, which will in turn hopefully make the experience for frontend users better as well
- separate code from data
- demonstrate plugins handling with Ghidra (demonstrated.)
- refactor the R library infrastructure, to demonstrate the transferrability/general suitability of the approach to other backends (demonstrated,
but not fully functional, further work may yield some useful feature creep - WIP)
- write actual RFC
- get RFC accepted
- apply to all the things.
- rooted-overlay started in, and was factored out of - the Ghidra plugin handling infrastructure
- nix-rer is the first POC/test of the initial implementaiton of rootedoverlay, and the results and missing features are being fed back into rootedoverlay
[RFC] Proposal: Engineering a Common "Plugin" Infrastructure
general library for “rooted” package+plugin systems. @84f14ba at time of post.
Ghidra demonstrator @b8c4562 at time of post.
R demonstrator (a mostly full refactoring of r-modules, very WIP) @25f87a4 at time of post.
- I’m looking for suggestions for better terminology
- The automatically imported, numbered layers approach is applicable to any system of overlays and perhaps could/should be separated out.
I’d be happy to discuss on freenode/#nixos-wg-ai as well (but you can find me on most of the other nix channels too).