History: Forum and Mailing List Synchronization

Source of version: 23 (current)

^This page is about the Tiki feature. Maybe you are looking for Tiki ((tw:Mailing Lists))?^


! Forum and List Synchronization

Related: (related(Mail-in))

This page describes how to set up a Tiki ((Forums|forum)) such that all web posts to the __forum__ go to a __mailing list__, and all e-mails in the __mailing list__ are posted into the __forum__.  It preserves e-mail threading, even on posts in the forums!  This allows Tiki to become both a web-based and email-based forum or ''group'' where users can use either email or web, or both, to view and submit messages.  

Note that the Tiki does not provide the mailing list server capability.  You will have to use some external service like __Mailman__, __YahooGroups__, __GoogleGroups__ or similar.  Tiki will use its sendmail access to send web-based forum posts to the mailing list.  Then you need a unique email account with POP3 (not browser) access for Tiki to retrieve email traffic from the list to post to the forum.  

!! Basic Setup
The portion of the ((Forum Admin|forum administration)) page you need to modify is as shown below. 

{img link="display2050" src="display2050" alt="forum_and_list_synchro_admin_v3.png 14.37 Kb" rel="box[g]" imalign="center" desc="Click to expand" align="center"}

There are actually very few options to set in Tiki.  But the options setup in Tiki and similarly named options in your Mailing List software can be tricky to get correct.  So read through everything to understand the implications before starting.

Before giving the details on each Tiki option, lets give a quick summary of the three email addresses of concern:

|| ::__Address Example__:: | ::__Quick Summary__:: 
__ListName@MyMailListServer.com__ | __To:__ email address for Tiki to submit Forum messages to the Mailing List
__ForumOut@myTikiSite.com__ | __From:__ email address for Tiki when submitting Forum posts to the Mailing List
__ForumIn@myMailHost.com__ | POP3 email account that is subscribed to the Mailing List and used by Tiki exclusively to retrieve messages into the Forum ||

Each Tiki option and its setting is covered next.  We will then follow by talking about the additional setup required in your mailing list software. 

|| ::__Option__:: | __::Description__::
"__Forward messages to this forum...__" | The email address used to submit messages to the mailing list.  For example, __ListName@myMailListServer.com__ or __GroupName@yahoogroups.com__.  Tiki will send a message __To:__ this email address whenever someone posts to the forum via the Tiki Web interface. This is the __outbound email address__.  ^__Tip__: Do not associate the Tiki Forum option "''Send this forums posts to this email''" with this mailing list in any way.  Emails sent to that email address are sent from the Tiki Admin email account in __link__ format and will cause a duplicate message back in the forum.^
"__Send mails even when the post is generated by inbound mail__" | If you enable this option, you will likely generate an undesired loop.  Best to leave unchecked.  When checked, will cause any post to the forum, whether via the web or from inbound email, to be posted back to the outbound email address.
"__Append a reply link to outbound mails__" | If checked, emails sent to the mailing list from the forum will include a web link to make it easy to post a reply using the forum web interface in Tiki.  Generally not needed if your mailing list software has __Reply-To:__ processing setup as desired and people prefer to use their mail reader to reply to messages.  But some Tiki registered users may want this option to more easily use the Tiki forum to compose their reply instead of the email program.
"__Originating e-mail address...__" | This is the __From:__ email address that a Forum post will be sent from.  That is, web-originated posts in the Tiki Forum generate an email from this address to the outbound email address given earlier. An account for this email address does not have to exist but SPAM processing may block the message if the domain or account of this __From:__ address cannot be validated. %%% %%%In this example, we are using: __ForumOut@myTikiSite.com__.  If meant to be a real address, you may need to use your Tiki mail servers domain (if different).  ^__Note__: this email address must be able to submit to the mailing list.  So the email address should be subscribed to the mailing list if closed. Unfortunately, web-based forum posts cannot simply be sent as if coming from the Tiki registered user account (that is, the registered users email).  All Tiki forum posts come from this same email address.^  
"__Add messages from this email to the forum__" | The mail server, user and password of a POP3 account that is __solely__ for retrieving email from the mailing list into the Tiki forum.  No one should ever login to this account except Tiki.  This email address should be subscribed in the mailing list and receive messages individually (no digesting). %%% %%% In this example we are using: %%% ''POP3 server'': __mail.myMailHost.com__ %%% ''User'': __ForumIn@myMailHost.com__ , and %%% ''Password'': (the password for that email account).||

!!Tiki Registered Users and Mailing List Subscribers

A registered Tiki user who makes a post using the web-based Forum will appear in the Mailing List message with their Tiki user name.  The __From:__ email address will be the same for all users posting as detailed above.  An email coming into the Tiki Forum from the Mailing List will be scanned.  If the __From:__ email address is the same as one of Tiki's registered users, then the forum post of that incoming email will show as coming from that registered Tiki user.  Otherwise, unrecognized __From:__ email addresses (or Forum users not logged in when posting) will appear as __Anonymous__.  So Tiki registered users can use email to read, write and respond to posts and it appears as if they did everything via the web-based forum directly.

!! Transparency of Mailing List and Tiki Forum

Tiki checks the incoming email account and processes all pending emails each time the forum is accessed in a browser.  So any Forum visitor will see the latest posts -- whether from a web-based Forum posting or via email in the Mailing List.  Nobody can check the Tiki Forum without Tiki first processing the email list for new entries.  Emails are sent out to the Mailing List immediately when a web-based forum post is made.  So this setup of two disparate services is somewhat transparent to the users of either mechanism (mailing list or Tiki Forum).  

But the forum will not update with new email unless someone checks the forum.  This only really matters if doing a site-wide search in Tiki before looking into the forum.  Recent emails to the Mailing List may still be pending and thus not found in a search until someone visits the forum web page first.  Only the (related(Mail-in)) option for Wiki and Article updates operates independent of user access to the Tiki but this service does not apply to Forums.

!!Mailing List Setup

The tricky part is to avoid mail loops during set up and operation.  Tiki Forum processing knows to ignore incoming emails coming __From:__ the originating email address.  But sometimes the sender gets changed by the mailing list software.  So setup can be tricky and dependent on your mailing list software and settings.

^Hint: While setting up, make the Mailing List be moderated.  At least until you're __sure__ that everything is working OK.  You may want to keep it moderated even after; especially if you allow anonymous posts in the Tiki forum.  This will help control the email coming-from and going-to the Tiki Forum during setup and break any unintentional loops.  Linking a Tiki Forum into an active, operating Mailing List is possible but difficult to get right without many test messages going to your active Mailing List members.^

Make sure the email addresses (__ForumOut@__ and __ForumIn@__ in the example above) are subscribed to the Mailing List.  __ForumOut@__ posts should be pre-approved (not moderated) and should not receive emails back from the list. Only moderate this address in the Mailing List if you allow anonymous posts in the Tiki Forum and do not want to clutter your Mailing List with anonymous posts.  If __ForumOut@__ is not a real account, and the Mailing List attempts to mail to it, the Mailing List may unsubscribe the address.   This email address is only for submitting messages to the list.  __ForumIn@__ should get emails directly and individually from the Mailing List; no digesting.  If you have the option, text emails are generally better than HTML going to this address.  This email address can be moderated in the Mailing List as it should never be used to post a message to the list; only receive messages.  No one else should ever use the POP3 account login associated with __ForumIn@__ to retrieve messages; else the Mailing List and Forum users may not see all the same messages.

The three different email addresses in the Tiki Forum setup can all be different or sometimes purposely the same.  But you cannot have all three be the same.  And never make the "Forward messages ..." (Mailing List address) and "Add messages ..." (POP3 account) be the same address.  In general, it is best they all be different.  But your setup may require you to make some the same to avoid looping.

Tiki Forum users will likely be confused by automatic messages generated by the Mailing List software such as password reminders, automatic response messages on submission, and bounce processing.  So if you can avoid those being sent to the Forum POP3 account, do so.  This is why having a separate account for incoming email (for example, __ForumIn@__ above) is helpful.  You can usually set the membership properties in the Mailing List uniquely for that email address.

Tiki, as well as most mailers, set the __Reply-To:__ field to be the same as the __From:__ field in the email header.  Some Mailing Lists reset the __Reply-To:__ field to be the Mailing List address.  Some users find this annoying as simple replies done in their mail reader meant to go back to just the sender are then broadcast back to the whole list.  But as Tiki Forum uses a generic __From:__ address not specific to a user, this rewrite in the Mailing list can be helpful.  A Mailing List user cannot privately reply back to a Tiki Forum poster and no error will necessarily be generated (nor should it else you may get automatically unsubscribed).

An option can be to use the Mailing List outbound address as the __From:__ address also to force replies back to the whole list whether the rewrite is on or not.  Another option is to make the POP3 account and __From:__ addresses actually be one in the same.  But this can also get you into trouble and begin to make the mailing list and forum get different messages.   So it is likely best to:
# Use three separate email addresses
# Setup the mailing list with __Reply-To:__ processing turned off (that is, left to be the sender's email address in the email list), and 
# Assure that emails to the __From:__ address are ignored and not automatically responded with an error message.
If your Mailing List software or the mail server modify the __From:__ field of messages generated by the Tiki Forum, those messages coming back into the Forum may be interpreted as new and different and added to the Forum as a new post.  In this case, messages originated in the Forum will get duplicated in the Forum.  The only way to break this loop is to work with the POP3 mail account setup or possibly Mailing List software to break the loop.  One method to try is to blacklist the __From:__ address used by the Tiki Forum in the POP3 account setup so Forum messages are ignored.  Another is to set the __From:__ and POP3 account email addresses the same and tell the mailing list software not to send posts back to the sender.  If none of these are working, you need to analyze the mail headers to see how your messages are being modified and adjust the settings to counteract this, as possible.

Each mailing list engine may require some other unique options.  Some common mailing lists and their setups are covered next. 

!! Using "Yahoogroups"
For yahoogroups ([http://groups.yahoo.com]), it is recommended that you subscribe your POP3 account as follows:

!!! Step 1: contact details
__Yahoo Profile__: your Tiki Forum name, 
__Email address__: The POP3 account email address. Here you have to add the same email address used before: __ForumIn@myMailHost.com__

!!! Step 2: Messages delivery
Select __Individual messages__ only (don't select "Daily digest", or "Special announcements", or "Only Web").

!!! Step 3: Mail preferences
Select __Traditional__ (instead of "''Complete (New and Improved)''"), for easier viewing in your Tiki (all ads come below the text message).

[note: we need more detail about the __ForumIn@__ account to subscribe.]

Press __Save changes__ when done, and that's it.

!! Using "Mailman"
__Mailman__ ("The" GNU Mailing List Manager) is free software for managing electronic mail discussion and e-newsletter lists, and is used all around the globe. More information: [http://www.gnu.org/software/mailman/] 

__How to configure it?__

[note: detail needed here yet.]

For other Mailing List programs, follow a similar approach, and if needed, request some help in [http://tiki.org/forums] ;-)


-= Alias names for this page =-
(alias(Mailing List)) | (alias(Mailing Lists)) | (alias(Forum and List Synchronization)) | (alias(Forums and List Synchronization)) | (alias(Forums and Mailing List Synchronization)) | (alias(Forums and Lists)) | (alias(Forum_and_List_Sync)) | (alias(Forum_and_List)) | (alias(Forum and List)) | (alias(Forum to List Gateway)) | (alias(Forum to Mailing List Gateway))