The history of blood and tears of an open source software author and email notification struggle

Preface:

On September 18, Zendao released version 7.3, which is the 65th open source version released by Zendao within five years, and it is also the "history of blood and tears" that we have struggled with email notifications for five years. In this version, we finally integrated a big move to completely solve the problem of email notifications. Let's sell it first, and then we will talk about what our ultimate move is in detail.

background:

Zentao ( http:/www.zentao.net ) is an open source project management software developed by our team , whose main orientation is R&D project management. The target user groups are mainly R&D teams, and the deployment scenarios are mainly private servers within the enterprise. This is the big background of our story. And then the tragic story begins.

Unbeatable Xiaoqiang: Email

One of the requirements of ZenTao software during use is to notify relevant personnel of various dynamic messages in the software. There are many ways to solve this problem: reminders of client software, reminders of QQ, reminders of WeChat, reminders of SMS, reminders of emails, reminders of desktops of browsers, etc. Each has its pros and cons, and then the email we've been fighting with for five years is here. Among the various notification methods mentioned above, email notification is the most widely used and most closely matches the user's usage habits. Speaking of which, some friends may say that our team emails are no longer needed. In fact, we still underestimate the tenacious vitality of email. As a basic service system that has existed since the early days of the Internet, the mail system has a wide user base. There have been all kinds of collaboration software trying to kill email, but unfortunately, there are no successful cases so far.

Naive thought: Programmers should get smtp right

One of the backgrounds of the story is that the main users of ZenTao are the R&D team, so we naively thought that configuring an smtp sending server should be a very simple matter for those who do software development.

Therefore, our initial version provides file-based configuration parameters. Users need to set the address and port of the smtp server themselves, and whether they need to log in. If they log in, they also need to set parameters such as username and password.

$config->mail->fromAddress = '';            // The from address.
$config->mail->fromName    = '';            // The from name.
$config->mail->mta         = 'smtp';        // The send mail type.
$config->mail->smtp->debug    = 0;          // Debug level, 0,1,2.
$config->mail->smtp->charset  = 'utf-8';    // Charset 
$config->mail->smtp->auth     = true;       // Need auth or not. true|false
$config->mail->smtp->host     = 'localhost';// The smtp server host address.
$config->mail->smtp->port     = '25';       // The smtp server host port.
$config->mail->smtp->secure   = '';         // The type to encode datas, 'ssl' or 'tls' allowed
$config->mail->smtp->username = '';         // The smtp user, may be a full email adress.
$config->mail->smtp->password = '';         // The smtp user's password.

As a result, we soon discovered that we were so naive. Too many users don't understand what is an smtp server, what is a port, what is encrypted, what is not encrypted, and must think of other ways.

The Windfall of Organizing Business Cards

Once when I was sorting out business cards, I observed the mailboxes left in the business cards and found that there are two types of mailboxes: public mailboxes and mailboxes with private domain suffixes. Public mailboxes such as gmail, qq mailboxes, etc. There are two types of mailboxes for private domain names: self-built email services and third-party corporate mailbox services. In this way, companies that build their own mail services actually only account for a small percentage. So there is our further solution: to solve about 80% of the configuration problems through templates. We have sorted out the templates of smtp server parameters of common domestic mail service providers such as Tencent mailbox, 163, 263, gmail, and Sina. And simplify the configuration parameters entered by the user to only need to enter an email address, and we will automatically infer the corresponding configuration parameters.

We will try to find the mx resolution record corresponding to the domain name, and then get the service provider used behind it, and then set the parameters according to the corresponding template.

At this point, for most users, the configuration of the mailbox can be simplified to only need to enter an address for sending letters, and then enter the password. Finally we were clean for a long time.

The situation is changing, and the problem will arise again

Solved the problem of configuration parameters, and the problems in the actual use process began to be prominent. These problems can be classified into the following four categories:

1. Internal environment restrictions: For example, domain name resolution cannot be done, the PHP environment lacks ssl support, the security level is too high, and so on.
2. Third-party mail service providers add a lot of additional restrictions, such as smtp service is disabled by default, and requires verification code to open it.
3. The slow sending speed causes problems that affect the operation experience. We tried to provide asynchronous sending function, which is another big pit.
4. The problem of the mail reach rate: Now Zen Tao, zentao has become a sensitive word, and the probability of being killed directly is very high.

At this time, we noticed the email notification service of sendcloud, and the big move began to be brewed.

Get out of the loop and solve problems

sendcloud is a manufacturer specializing in providing email sending services, and I think there should be many manufacturers using their services. Zendao's saas service also uses sendcloud to send emails, and the effect is still great, with fast speed and high reach rate. For the sake of anti-spam, sendcloud's mail service still needs a lot of settings, and I think friends who have used it should have experience. These settings still have certain challenges. It is not appropriate to recommend directly to our users. The mail sending service of sendcloud is mainly for public network users, so their requirements for anti-spam will be relatively high. But if it is specific to an enterprise's internal management scenario, these restrictions can be ignored.

So I tried to send an email to my classmates from sendcloud, explaining our application scenarios and demands, and asked them if it was possible to create a notification service specifically for internal application scenarios of the enterprise. No other configuration is required. Users only need to activate the service, set the whitelist for sending letters, then get an id and private key, and then send letters through the http interface. I am very happy that the students of sendcloud responded quickly, and soon their notice.sendcloud.net service was launched. We also quickly integrated this service into ZenTao. So there is the big trick mentioned at the beginning of the article:

When sending a letter, you can choose whether to send a letter via smtp or sendcloud.

The story has come to an end today, and it can finally come to an end. But the story will not end, the battle with the mail will continue, and the story of tomorrow will continue to be told tomorrow. And I believe that with the rapid development of computers, and with it, more and more complex systems, and with it, less and less hands-on ability of users. The story will become more and more exciting, and the exciting level will definitely exceed " My machine can't ping the smtp server, how can I send a letter ?"