Source: Redislabs
Author: Pieter Cailliau, LucaAntiga
translation: Kevin
Brief introduction
Today we released a preview version of RedisAI, pre-integrated [tensor] werk components. RedisAI is a service tensors tasks and can perform tasks Redis depth learning modules. In this blog, we will introduce the function of this new module, and explain why we believe it will subvert machine learning (ML), the depth of learning solutions (DL) is.
Produced RedisAI are two reasons: First, the high cost of migrating data to the host performs AI model and experience a great influence on real-time; secondly, Serving AI model has always been a challenge in the field of DevOps. Our purpose of constructing RedisAI is to allow the user can not move Redis multi-node data of the situation, but also a good service, update and integrate their own models.
Location data is important
In order to demonstrate operation of the machine learning, deep learning the importance of the position data model, we give an example of a robot chat. Chatbot usually recurrent neural network model (RNN), to address one (seq2seq) User Q & A scene. More advanced model uses two input vectors and two output vectors, and the digital intermediate state vector in a manner to preserve the context of the conversation. Model uses the last message as the user input, dialogue among representatives of state history, and its output is a response to user messages and a new intermediate state.
In order to support interactive user-defined, this intermediate state must be saved in a database, so Redis + RedisAI is a very good option, there will be the traditional program and RedisAI program to make a comparison.
1 , the traditional program
使用 Flask 应用或其它方案，集成 Spark 来构建一个聊天机器人。当收到用户对话消息时，服务端需要从 Redis 中获取到中间的状态。因为在 Redis 中没有原生的数据类型可用于 tensor，因此需要先进行反序列化，并且在运行递归神经网络模型（RNN）之后，保证实时的中间状态可以再序列化后保存到 Redis 中。
考虑到 RNN 的时间复杂度，数据序列化/反序列化上 CPU 的开销和巨大的网络开销，我们需要一个更优的解决方案来保证用户体验。
2、RedisAI 方案
在 RedisAI 中，我们提供了一种叫 Tensor 的数据类型，只需使用一系列简单的命令，即可在主流的客户端中对 Tensor向量进行操作。同时，我们还为模型的运行时特性提供了另外两种数据类型：Models 和 Scripts。
Models 命令与运行的设备（CPU 或 GPU）和后端自定义的参数有关。RedisAI 内置了主流的机器学习框架，如 TensorFlow、Pytorch 等，并很快能够支持 ONNX Runtime 框架，同时增加了对传统机器学习模型的支持。然而，很棒的是，执行 Model 的命令对其后端是不感知的：
| AI.MODELRUN model_key INPUTS input_key1 … OUTPUTS output_key1 .. |
这允许用户将后端选择（通常由数据专家来决定）和应用服务解耦合开来，置换模型只需要设置一个新的键值即可，非常简单。RedisAI 管理所有在模型处理队列中的请求，并在单独的线程中执行，这样保障了 Redis依然可以响应其它正常的请求。Scripts 命令可以在 CPU 或GPU 上执行，并允许用户使用 TorchScript 来操作Tensors 向量，TorchScript 是一个可操作 Tensors 向量的类 Python 自定义语言。这可以帮助用户在执行模型前对数据进行预处理，也可以用在对结果进行后处理的场景中，例如通过集成不同的模型来提高性能。
RedisAI 的数据类型和后端概览
我们计划未来通过 DAG 命令支持批量执行命令，这会允许用户在一个原子性操作中批量执行多个 RedisAI 命令。例如在不同的设备上运行一个模型的不同实例，通过脚本对执行结果做平均预测。使用 DAG 命令，就可并行地进行计算，再执行聚合操作。如果需要全量且更深的特性列表，可以访问 redisai.io。新的架构可以简化为：
模型服务可以更简单
In a production environment, use Jupyter notebooks to write code and deploy applications are not the optimal solution in Flask. How to determine the user's own resources is the best of it? If the user host is down, the intermediate state of the bot what happens? User-create the wheel may be repeated to achieve the existing Redis functions to solve the problem. In addition, due to the complexity of the combination regimen often than expected, stubbornly adhere to the original solution will be very challenging. RedisAI by Redis enterprise-class data storage solutions, supporting data type Tensors, Models and Scripts such as deep learning needs, to achieve a good depth of integration Redis and AI models. If you need to expand the computing power of the model, simply for Redis cluster expansion can, so users can add as many models in a production environment, thereby reducing infrastructure costs and total cost. Finally, RedisAI well adapted to the existing ecological Redis allows users to perform scripts to pre-and post-processing of user data, the data structure can be used RedisGear do the right conversion, can be used to keep data up to date RedisGraph state.
Conclusions and follow-up plans
1, in the short term, we want to use RedisAI in the case supports three major back-end (Tensorflow, Pytorch and ONNX Runtime), and stabilized as soon as possible and reaches a steady state.
2, we hope that these can be dynamically loaded the back end, users can customize the designated back-end loaded. For example, this would allow the user to use edges with Tensorflow Lite treated cases. 3, plans to achieve the automatic scheduling function can be achieved automatically merge different queues in the same model. 4, run statistical models RedisAI will be used to measure the performance of the model.
5, to complete the DAG characteristics explained hereinbefore.
More high-quality IT middleware technology / original / translation of the article / data / dry goods, please pay attention to "middleware little brother" public number!