Attention to people working on topics related to indexing and search: you can get your (research) proposal funded (equity free) through the European Commission’s NGI Search program. Individuals can get up to 50,000 EUR, and organizations can get up to 150,000 EUR.
For further details about scope and eligibility: Main - NGI Search
I believe indexing package ecosystems is quite important in order to better integrate foreign ecosystems with nix.
I have done some indexing projects in the past like pypi-deps-db or nix-pypi-fetcher which are the basis for approaches like mach-nix
Also the haskell infrastructure in nixpkgs depends on indexes like all-cabal-hashes.
Having a more uniform way for indexing, storing, and accessing this kind of data would be of great use, I believe.
There is already https://libraries.io, but the data there is not suitable for reproducible builds as it doesn’t contain checksums for sources etc. and it doesn’t have a great way of exporting subsets of the data, AFAIK.
Another problem is, that as of now, nix itself cannot really be used for dynamic dependency management. Currently all language2nix approaches rely on external tools to resolve dependencies and generate lock files.
For example, one can build a nodejs project with buildNpmPackage or dream2nix, but in order to add dependencies or update them, one still needs to call out to npm.
In many environments nix is only suitable as an addition to other package managers, not as a replacement.
If we want to have a more nix-native approach to dependency management itself, we need reproducible metadata access for more external ecosystems.
Having that, would allow us, for example, to implement reproducible dependency resolution across ecosystems without calling out to many different tools.
The indexes mentioned above, are non-standard, ad-hoc solutions for specific ecosystems.
Projects with the goal of unifying these indexing approaches sound like they could align well with NGI Search and could be of great use for the nix ecosystem and potentially other ecosystems.
Dave that’s a great Idea,it’s not all looking for cat pictures at the end of the day.
Indexing all the ecosystems properly for reproducible code sounds like fun and have the data in sane format for machine processing, sounds ideal.
@DavHau GitHub - package-url/purl-spec: A minimal specification for purl aka. a package "mostly universal" URL, join the discussion at https://gitter.im/package-url/Lobby et al. should receive venerable mention, as well.
The forming Software Supply Chain space in general might very well come up with suitable formats, so we’ll have an opportunity to pair with other ecosystems on this with the goal to improve adoption.
Instead, I believe we need some way to extend flake inputs resolution with a ‘plugin system’, I realized that after reviewing a PR to add sourcehut to flake inputs. Turns out that we have other git servers, other non git protocols, others non github apis. And why not a dependency resolution, like flake does, but only if it is another flake.
Something like this but user friendly
https://elvishjerricco.github.io/2018/06/24/secure-declarative-key-management.html#sharing-secrets
It’s great that people have thoughts about it, but discussing it here probably isn’t going to help getting those EU grants 