Import the mailing list?

Meta Moderation
12

The mailing list setting can be turned off by the time the import is being made via API, and then be turned on later?

13

https://docs.discourse.org/#tag/Site-Settings-Email%2Fpaths%2F~1admin~1site_settings~1disable_emails%2Fput

14

Sounds good. Who is volunteering for writing the import script?

15

Well, what is needed?

16

We need 2 scripts that can import messages using only the Discourse API:

  • one for the mailing-list archive
  • one for the Google Group
    Ideally both would be tested on a local Discourse instance before running them against production. The best way to run a local Discourse instance at the moment is to use this Docker image: Docker
17

Maybe someone has an email archive from the mail list and Google? Someone who has subscribed from the beginning and is making a backup:)

18

I spent some time this weekend poking at this and decided it might be easier to coordinate/collaborate on this endeavor via github.

My ultimate plan is that the repo serves as a comprehensive solution for the NixOS Discourse admin to follow which clearly documents how to perform the entire importing process along with any custom tools created.

I found the nixos.org mailing list archives for 2004-2017 and added a Bittorrent file to the repo for anyone interested.

There are also features/issues to consider when designing the importer, and I anticipate it’ll be easier to discuss them in the issue tracker than here.

2 Likes
19

There is 0 peer for the torrent. Why not IPFS or use Git LFS?

20

How big is the archive? Depending on the archive file size it could also be uploaded to GitHub as part of a git tag release. Or even better, added to the git repo.

21

Answer:

$ curl -sf https://nixos.org/nix-dev/ | perl -n -e'/ (\d+) KB/ && print "$1\n"' | awk '{s+=$1} END {print s " KB"}'
10729 KB
22

Weird, I have two torrents seeding from different States. Was able to upload from one to the other as a test.

I didn’t want to commit an 11 MB file to the repo.

I’m not familiar with IPFS or Git LFS. I installed IPFS and tried to upload the archive, @apeyroux. I was able to use the following command to retrieve it.

ipfs get -o nix-dev-mailing-list-2004-2017.tar /ipfs/QmPppdYKjHhp8qgueQampBdNchCS59k2TRAii7gQG8B3Y7

Let’s see if that works. Will read up on Git LFS and see if I can get that to work, too.

1 Like
23

Thanks, I could download it

mkdir nix-dev && cd "$_" && ipfs cat /ipfs/QmPppdYKjHhp8qgueQampBdNchCS59k2TRAii7gQG8B3Y7 | tar -xv

or get and pin .tar

ipfs get --output=nix-dev-mailing-list-2004-2017.tar /ipfs/QmPppdYKjHhp8qgueQampBdNchCS59k2TRAii7gQG8B3Y7

24

Is there a way to also migrate subscriptions? or at least send out invitation.

25

It can be useful: Generate an invite link but don’t send an email

26

Oh, that’s awesome, thank you!

27

I found that tool. Can it be useful?

28

If someone is interested, I’m downloading the old mailing list in text mode.

29

Huge bump, sorry, but I think it would be still nice to do this? Just glancing at Migrate a mailing list to Discourse (mbox, Listserv, Google Groups, etc) - sysadmin - Discourse Meta maybe there is new stuff for this in the meantime?

30

I have downloaded all the contents of those mailing lists. Is someone still interested?

31

Here is it: