Import the mailing list?


#1

It would be nice if old posts would be still searchable.
There exists a guide on how to import mailing lists archives here:

However I am not sure if we can import it easily on that hosted instance.


#2

I’d also love to see this. Especially since last time we weren’t able to import the old emails - this would probably encourage even more users to use discourse as all mail history will be searchable.


#3

Do we still have access to the old mailing-list archives?


#4

I bet someone backuped them or most of some.


#5

here we go: https://www.mail-archive.com/nix-dev@lists.science.uu.nl/


#6

I contacted Discourse’s support, let’s see what they say


#7

I hope this will be possible - plus, and I consider this important, that the ML does not get re-send on import (I mean if I enable mailinglist mode here and you import all the mails from the mailinglist, I get them again… that’d be bad!)


#8

The Discourse team has been generous to provide us with a free hosted instance but it means that I don’t have command-line access to do the import. When contacting support about this, this is the answer that I received back:

According to https://payments.discourse.org/pricing this would cost us 3600 USD / year.

Another solution would be to host Discourse ourselves.


#9

Anyone have a.tar.zg email archive?


#10

And use webservice https://docs.discourse.org/#tag/Topics ?


#11

Using the API to import would probably spam the users that have the “mailing-list” setting on


#12

The mailing list setting can be turned off by the time the import is being made via API, and then be turned on later?


#13

https://docs.discourse.org/#tag/Site-Settings-Email%2Fpaths%2F~1admin~1site_settings~1disable_emails%2Fput


#14

Sounds good. Who is volunteering for writing the import script?


#15

Well, what is needed?


#16

We need 2 scripts that can import messages using only the Discourse API:

  • one for the mailing-list archive
  • one for the Google Group
    Ideally both would be tested on a local Discourse instance before running them against production. The best way to run a local Discourse instance at the moment is to use this Docker image: https://hub.docker.com/r/discourse/base/

#17

Maybe someone has an email archive from the mail list and Google? Someone who has subscribed from the beginning and is making a backup:)


#18

I spent some time this weekend poking at this and decided it might be easier to coordinate/collaborate on this endeavor via github.

My ultimate plan is that the repo serves as a comprehensive solution for the NixOS Discourse admin to follow which clearly documents how to perform the entire importing process along with any custom tools created.

I found the nixos.org mailing list archives for 2004-2017 and added a Bittorrent file to the repo for anyone interested.

There are also features/issues to consider when designing the importer, and I anticipate it’ll be easier to discuss them in the issue tracker than here.


#19

There is 0 peer for the torrent. Why not IPFS or use Git LFS?


#20

How big is the archive? Depending on the archive file size it could also be uploaded to GitHub as part of a git tag release. Or even better, added to the git repo.