Discourse patch submission is not fit for purpose

We don’t need to mirror the Nixpkgs git repo, right? We just need a
mailing list.

I have a mild (because it’s only a mailing list, and can’t be that
difficult to get right) preference for not using SourceHut. In my
evaluation, I have found it to not work particularly well (not
unreasonably, given it’s in alpha), but what I’ve found more concerning
is the attitude of the author towards bug reports. For example, here he
tells me that it is my problem that his software doesn’t support the
common practice of posting revisions of a patchset in reply to the
first: https://todo.sr.ht/~sircmpwn/lists.sr.ht/128

That’s not to say we couldn’t mirror a mailing list to SourceHut –
that’s one of the nice things about mail, and I gather Drew is working
on support for it.

One option: I already host mailing lists for Spectrum (see
https://spectrum-os.org/participating.html#mailing-lists), and am
committed to and funded for doing so for at least the next year. I
would be happy to add an additional list for Nixpkgs. It could be
mirrorred and easily replaced should I suddenly disappear or whatever,
and should SourceHut look more convincing in future it would be easy to
migrate. Available on that server are Mailman, public-inbox (for
archiving and web browsing), and Hyperkitty (a nice modern web UI
suitable for people who don’t want to use mail).

I don’t think we should try to mirror mailing list discussion to
Discourse either – it would end up being ugly on both sides, I think.
It might be better to create a simple web application that would show
proposed changes from both the mailing list and GitHub, that reviewers
could follow.

@qyliss looking at the bug report, it seems like something got lost in translation. Drew is developing a whole new developer platform, and handling all the customer support himself. It’s possible he might have rushed and not properly understood what you meant exactly. The whole platform is based on the idea of patch-based development so it would be surprising if common scenarios weren’t something that was intended to be handled.

I don’t have a strong preference either way. Whatever solution is selected in the end, I think the important criteria is that there are some people with nixpkgs push rights monitoring and handling the submissions. The disadvantage of both options right now is that they don’t integrate with the existing ecosystem and are yet another platform that needs an account.

I don’t have a strong preference either way. Whatever solution is
selected in the end, I think the important criteria is that there are
some people with nixpkgs push rights monitoring and handling the
submissions. The disadvantage of both options right now is that they
don’t integrate with the existing ecosystem and are yet another
platform that needs an account.

I don’t think that either would need an account – usually you can
subscribe to a mailing list just by sending an email, and with mine at
least you don’t even need to be subscribed to post.

But, yes, I agree with the second point. So, Nixpkgs committers, please
reply to this post if you would be willing to monitor such a list,
provide review, and push contributions to Nixpkgs.

I would totally do that!

I don’t think we should even accept patches over email.

Patches over email doesn’t fit our current workflow at all and it requires someone else to take your patch and create a Github pull request for the patch to enter our normal review and test flow.

While I understand and agree that Github has it’s set of problems having a completely separate review flow without any testing infra is something I’m strongly opposed to.
Maintaining nixpkgs is already hard enough.

I think it’s time to formally say patches over email is not an acceptable form of contribution for Nixpkgs.

2 Likes

So testing is your only concern? If so, it’s solvable.

One thing that seems fairly obvious to me but maybe should be stated is that the person who acts as a proxy also becomes the owner of those changes. Somebody who submits patches by email cannot become an owner as they will not get the notifications from GitHub or participate in the ecosystem meaningfully (eg: no RFC contributions). In that sense, this list should be transparent to the normal workflow. It’s an unofficial add-on that caters to the needs of certain contributors.

As long as there are people willing to do the work, I don’t see much harm in having this. That’s also the beauty of the nix community, we have a good diversity in types of users.

One thing that seems fairly obvious to me but maybe should be stated
is that the person who acts as a proxy also becomes the owner of those
changes. Somebody who submits patches by email cannot become an owner
as they will not get the notifications from GitHub or participate in
the ecosystem meaningfully (eg: no RFC contributions). In that sense,
this list should be transparent to the normal workflow. It’s an
unofficial add-on that caters to the needs of certain contributors.

That’s true, although the person who wants something from the original
submitter could probably just email them themselves if they wanted to
(and get a faster response).

They wouldn’t have to, though.

It’s not just about testing. It’s the whole pull request life cycle that would have to be adapted for email.
It’s not an insignificant amount of work if you aim for the same quality as we do in Github pull requests.

I think it’s a fundamentally bad idea to have multiple contribution and review flows, having this will create a second class of contributors who cannot participate fully in the community.

The same, as in «and make sure it is no better»? In email I get all notifications but the patch context GitHub attaches is often insufficient; in the Web UI finding out all that has happenned since some time is a lot of work (and the full context might have been destroyed). Reasonable people manually replying to emails with patches usually end up with easier to read discussion…

2 Likes

I think it’s a fundamentally bad idea to have multiple contribution
and review flows, having this will create a second class of
contributors who cannot participate fully in the community.

We already have this, and always have – I’m not trying to introduce
it, I’m trying to improve what we have. And the alternative is a second
class of potential contributors who are locked out of contributing at
all.

I am just wondering maybe is worth considering RIOT as a platform ?
https://matrix.org/

I am just wondering maybe is worth considering RIOT as a platform ?

I don’t think so. Riot is a chat app. We’re looking for something for
long-form patch submission and code review. The options discussed so
far are already widely used for this, are integrated into Git, etc.

1 Like

I’d be happy to review patches from mail, and my personal patch submission flow already aims to let me ignore GitHub’s PR submission flow in favour of one that lets me pretend I’m dealing with mail, without making that fact particularly visible to others.

Since GitHub regularly destroys patch history, I already use tools meant for the email flow like git-series in order to track the history of my PRs.

I’m not too convinced of the value of sourcehut, especially when public-inbox and regular mailing lists already exist, provide cloneable archives and highlighted patches, and are already in use by the kernel and many adjacent communities.

1 Like

@edef are you okay having your email address published publicly?

Either create a mailing list or give me your email and I will transform the patches section of discourse with a message redirecting the user to it.

Sorry for necrobumping the topic but the email workflow for submitting patches still pops up here and there and we never really solved it (unless I missed something), for instance today a potential patch appeared on reddit :

Yeah I think we need some way to handle that. Personally I do not mind
receiving patches via Email. Most of my daily workflow evolves around
email. Such as reading these threads that are just mails in my inbox.
Just like another other mailinglist or GitHub Repo that I am following.

A few of my friends do not ever want to have a GitHub account
(especially after the recent acquisitions) and thus rather submit
throught me or via some ML.

It probably goes a bit too far for our topic here but there is also
discussions around how to modernize the entire mail based workflow
without having a vendor lock-in over on the public-inbox ML [1].
Unfortunately, that discussion didn’t go far.

I was also thinking about hosting a patchwork [2][3] instance for this
kind of activity. While I can certainly do some work around accepting
patches (and translating them into PRS & commits) I wouldn’t start this
without a few more comitters that are willing to help out.

The workflow around that kind of ML archive (unless you have all the
mail locally) could look similar to this:

# create a new branch
nixpkgs/ $ git checkout -b some-patch origin/master
# apply the patch (series)
nixpkgs/ $ git am < $(curl https://something/patch/123/mbox/)
# review & test
nixpkgs/ $ git log / tig / nix-build …

This really isn’t that much different from checking out a PR (minus
being on the right commit when you apply the series).

Afterwards you can just open a PR or commit as usual or simply reply to
those patches via Email if you want to request changes / provide
feedback.

The big problem here is probably not sovling this (patch receiving &
applying) technically but how to handle feedback and CI results in those
cases. As we are pretty much stuck on GH right now. The troubles here
could be that you first argue with the submitter via E-Mail, do multiple
iterations of patches until everyone is happt. Afterwards you put that
PR up on GH and then some more feedback arrives. It would probably not
be great if those that accept that patches then have the burden of
synchronzing between GH and responding to feedback. GH doesn’t even
allow unregistered users to subscribe to updates on a PR/issue.

On the other hand, if the patches we are receiving via Email are usually
small, self-contained and do not cause long dicussions then I don’t see
why this wouldn’t work.

[1] https://public-inbox.org/meta/CAMwyc-Td-AKNaLgxEw91ibBAkN-DdP1xhRSfs-oLkyh73yRQUw@mail.gmail.com/
[2] http://jk.ozlabs.org/projects/patchwork/
[3] https://lore.kernel.org/patchwork/project/lkml/list/

There are 5 patches in https://discourse.nixos.org/c/dev/patches/19, I recommend someone handles that and we optimize once the pain kicks in.

IMHO the pain already kicks in since discourse isn’t a proper mailing
list. It mangles with all the inputs.


-> missing attachment, not really actionable. Copying a patch from the
modified mail output is also not fun.


-> the pastebin that was used to circument discourse shortcomings
expired


-> has been handled by ryantm


-> more corrupt patches due to discourse

Of course we can now try to repair all those issues and still call it
not painful. At least in my case those patches also arrived via email
but were converted into some markdown gibberish. Not very helpful.

Patch submission to our project should just work like everywhere else
(where mail is supported): git-send-email shouldn’t be rejected as
invalid and should not be mangled by the ML software (discourse in our
case).

1 Like

In that case we can create a team of a few volunteers that are willing to review these patches and document whom to send the email to. For 5 patches there’s really no need to go further than that.