It’s been eight years since Siri was released. Why is Chatbot still pitted? [Data set download attached at the end of the article]

https://mp.weixin.qq.com/s/77SoSb-bK-hE3UM2VOOH0Q

By 超神经

场景描述：聊天机器人虽然众多，但在目前来看，它们的功能都还不够理想。如何打造出一款好的聊天机器人，是期待技术上的颠覆，还是在设计上考虑的更加全面。不妨听听资深人士怎么说？

关键词：聊天机器人  NLP  对话系统

As early as 2011, Apple released the intelligent voice assistant Siri. Apple, which had great expectations for this product, had to face the awkward use of Siri. Siri either doesn't understand what the user is saying, or responds to the user with a bunch of unknown content. Now that Siri exists, more users are molested/ molested.

On the market, there are not a few chatbots, but there are very few products that can penetrate the hearts of the people. There are many chatbot products that are fresh for a while, slowly revealing the "mental disabilities" and "chicken ribs".

For example, on the Facebook Messenger application, the voice assistant M went offline after three years of operation, and Microsoft Xiaoice, which has been upgrading, has the most commonly used function staying in the painless "awkward chat".

Facebook M was shut down in January 2018

The reason for this phenomenon is that, on the one hand, people’s expectations for Chatbot are low. On the other hand, even with the most advanced technology, it is still difficult to fully understand the human dialogue system. The different emphasis on the design concept resulted in the final divergence.

So how can we make a satisfactory Chatbot? A senior person summarized a set of "USED" framework.

The Chatbot used is the best

Shin Wee CHUANG

Shin graduated from MIT and worked for Microsoft and Standard Chartered Bank (SCB) successively. He is a veteran in the field of financial technology.

He was awarded the "China Top 50 Corporate Innovators" award by CBN Weekly in recognition of his numerous digital and marketing initiatives in China SCB digital marketing. He currently runs Pand.ai, a technology company he founded, dedicated to building an enterprise-level intelligent Chatbot.

In an article "A good chatbot is a USED chatbot", Shin Wee describes how to build a smart chatbot. The following is a compilation of the article.

Since the establishment of a smart Chatbot for financial institutions a year ago and the establishment of Pand.ai with the co-founder, a question we have often been asked is: How good is the Chatbot you developed?

Trying to answer this question is to explain how we use deep learning natural language processing (Deep NLP) to extract the semantics of user input, so that Chatbot can better "understand" the question and provide more accurate answers.

Here, I will discuss the thought process of chatbots that satisfy customers. We are now integrating these views into a product framework to form our core design principles, hoping to provide some help to those who want to build or use AI Chatbot.

This frame contains 4 letters: USED, where USED stands for:

U: It Understand me can read people

To build a good chat, the robot must first understand the conversation, otherwise, it will not respond as expected.

Most Chatbots rely on the technology of "pattern matching", which is effective for "understanding" common sentences, such as "how are you?", "what is your name?" and so on, but when dealing with complex sentences or very common Sentences seem to be laborious, which is one reason why some "mentally retarded" customer services exist.

To build a good chatbot, an NLP (Natural Language Processing) engine is a must. There are already some such engines, which are analyzed and processed through cloud services, and can be easily used in a variety of chat tools.

In addition, some companies provide more professional NLP engines for specific vertical industries or markets, which can distinguish the nuances in the language, such as the terms or slang included.

If you don't have much background or don't want to spend time building your own, the best way is to choose a suitable and powerful NLP engine. There is no obstacle for Chatbot to understand people.

S: It Serves me has service

A good Chatbot needs to perform a certain function in a specific scenario. So the most important thing is to make the robot familiar with the details in this field.

For example, if you want to deploy a Chatbot for the sales team to help them work better, you must prepare all related products and content, and create an ideal conversation in a structured format.

In addition to the basic question and answer, you also need to consider adding a quiz component to the Chatbot, so that it can help sales staff update their understanding of the product.

E: It Engages me can move people

In the initial stage of chatbot promotion, allowing users to interact with robots is a challenging task. After all, it is not easy to let users change the way they are used to, accepting new things.

Therefore, an effective push strategy can be formulated to help users get used to new tools.

Of course, pushing does not mean that you have to send boring product descriptions, but to do it carefully. For example, a simple Father’s Day greeting message to the fathers in the group will be a very heartwarming way.

D: It Delights me will bring fun

At present, most chatbots are developing in a playful and cute way. After all, products that bring happiness are often more attractive. However, with the emergence of more and more chat bots, if they only tremble and be clever, it may no longer be realistic to expect users to be happy all the time.

Find a way that chatbots can bring fun. But at the same time, pay attention to the situation to avoid embarrassment in serious situations.

Fortunately, language is not the only weapon of chatbots. For example, burying an Easter egg in a conversation may bring more surprises than any witty words with imitations.

The core of the golden rule

This "USED" framework can be used as an enterprise-level chatbot building guide, but in fact, its core is to fully consider its practicality.

There are many KPIs that can measure the quality of a chatbot, such as response time and accuracy, but the most important point is whether it helps achieve business goals.

Whether you want to use chatbots to improve customer service and sales efficiency, or generate potential customers, as long as it is a method that can achieve business goals, it can be used. For example, the following KPIs: the
number of conversations (not the number of messages);
the percentage of monthly active users (MAU, or DAU "daily active" and WAU "weekly active"); the
number of sessions per active user

The development of Chatbot does not just rely on technology

It is not difficult to develop a Chatbot, even if it is an AI-driven chatbot, but if you want to build a good, intelligent, and widely used Chatbot, you need to consider more than just technology.

After experiencing an upsurge in the development of Chatbot, it gradually became more rational. The development of NLP has never been so fast in computer vision or speech recognition. But to make a good Chatbot, perhaps the most important thing to consider is the design concept that makes it useful.

Because according to the current technological development, in the short-term future, the barriers for Chatbot products are data and design, not technology.

And maybe when building a Chatbot, you can give priority to service, and then the dialogue system-just like people, they first have ideas in their heads and then use dialogue to express them.

There is such a point of view: a product is a combination of many technologies, and only by having a correct understanding of each type of technology can a good product be made. "After all, we are still far from real artificial intelligence, and only what is available is valuable."

Super Neural Dataset

Hyper-Neural HyperAI collects and organizes hundreds of public data sets around the world, and provides domestic mirror downloads, and provides free services to scientific research institutions and developers.

Recommendations for data sets related to intelligent question answering:

Yahoo! Answers Q&A Data Set: The data set is 10 main classification data from Yahoo! Answers Comprehensive Questions and Answers1.0 data set. Each category contains 140,000 training samples and 5,000 test samples. Published by Cornell University, the file size is 304.72 MB;

SQuAD Stanford Questions and Answers Data Set: It is a reading comprehension data set composed of questions asked by mass workers in Wikipedia articles, where the answer to each question is a piece of text or span from the corresponding reading paragraph, in more than 500 articles There are more than 100,000 Q&A matches. Published by Stanford University in 2018, the file size is 34.09 MB.

For more intelligent conversation data sets (such as CMU real question and answer pair data set; Microsoft Maluuba NewsQA machine reading comprehension data set), please visit https://hyper.ai to download.