Machine Learning (ML) test data from a laptop or a science laboratory to the production process is not a lot of people have had the experience. Data scientists are often entrusted with this difficult task, because they know the machine learning algorithms, and it may be their first proposed. At the same time, there have been many challenges around the production environment deployment AI Artificial Intelligence solutions. In this article we will focus on two aspects of the subject matter described:
Theme 1, machine learning how to build a production application from the experiment turned
Theme 2, 7 artificial intelligence trends and AI learning how to operate the machine and collaboration.
First, the production of machine learning how to build an application from the experimental steering
How to start a successful business life cycle production of machine learning (MLOps), transferred to a minimum the feasibility of the product the same algorithm (MVP) by ML experiments in production services from one promising. MVP is common in product development, because they can help customers quickly obtain product / service, with enough features make it feasible, and to promote the next version based on the use of feedback. In the context of machine learning, MVPs help key demand of separate production of ML service, with minimal effort and help deliver it.
MVP step in the production of machine learning we describe the data scientists and organizations with hundreds of use cases and experience drawn from the past few years. As shown below:
Figure 1: Experimental production from developer to the ML path
Step 1: Make sure your use case: What would you like to do?
This may seem obvious, but the first step is to understand what the minimum requirements of your business applications is, and the gap between your experiment and minimum requirements. For example, if the characteristics of the features you assume that the available experimental than the business application to provide more, then this gap may affect production. The best way to find this gap is to support the definition of machine learning applications for your business applications.
Here are some questions you need to answer:
• ML from the point of view, this ML application will help solve any business problems?
• What ML app needs to predict what it will receive input??
• Is there enough data to train the model and measure the effectiveness? are these data clean, accessible and so on? experimental data may be used for manual cleaning. Production training data also needs to be cleaned.
• Are there preliminary experiments (in the developer / notebook / laptop environment) shows a promising way to deliver the necessary algorithms to predict / quality?
• How ML application needs and business applications (REST, Batch, etc. )integrated?
Once these questions have been answered, you can generally understand what needs to ML application in MVP. This laid the foundation for steps 2 and 3.
Step 2: set out a list of state: What do you have?
Once the use cases, the next step is to integrate the initial state, so you can map the journey to the destination. A typical feature activation state comprises the developer from the desired level of ML prototype application member:
a software program • scientists data environment, such as a notebook Jupyter, R developer environment, Matlab like. The code is usually performed initial (promising) machine learning models and experiments.
• This code data is already in a lake or other data in the database to run through one or more sets. These data are part of the customer database lakes and normal data center infrastructure environments. These data sets may exist in notebook computers, data may need to be moved to the lake.
• Run the code in these training developer environment, sometimes there are examples of models.
In many ways, this is similar to the state of startup software prototype to other (non-ML) domain. Like other software, prototype code may not be using the production version expected in all of the connectors, scale factor and enhancements to write. For example, if the production version needs to read data from the cloud object store, and your experiment need to read data stored on the laptop, you need to add the object store connector into the production pipeline code. Similarly, if you quit the experiment code when an error occurs, it is possible to produce unacceptable.
There are also specific to the machine learning challenges. For example:
• generating the model presented here may require introducing to guide the production tubing.
• You may need to add specific ML code is detected, for example, report statistics ML, ML generate specific alarms, collecting detection (statistics performed) for long-term analysis and other purposes.
Step 3: Define your MVP product
now you're ready to define MVP: The first basic service you will use in production. To do this, you need to determine first production location, which is the first location code runs.
This largely depends on your environment. A short-term option might be to provide your data center infrastructure for other applications (non-ML, etc.). You may also have a long-term view, including integration with software or cloud services strategy and other aspects.
In addition to determining the first location, you also need to address the following issues:
• Lake access data can be generic, unless the organization (especially enterprise) set specific access restrictions applied to the analysis of data used.
• You must install the machine learning engine (Spark, TensorFlow, etc.). If you use a container, which can be very versatile. If you use the analysis engine, it may have a high ML features. Need to find and contains all dependencies (which you need to run the pipeline library access, etc.).
• the need for analysis engine and container size adjusted to ensure that the scope of the initial performance test and debug is reasonable. Testing and commissioning will be iterative.
• need to define the upgrade process. For example, if you decide to upgrade your plumbing code, and it needs a new library not previously installed, you will need to consider how to deal with it.
Step 4: preparation for the production of the code
now, you need to consider your experiment (if any) which requires the use of code in production. If you do not plan to retrain the model in production, the code then you need to consider just for reasoning. In this case, a simple solution might be to deploy a predefined inference pipeline provided by the supplier.
If you plan to be in a production environment retraining or have a custom demands, but pre-built inference pipeline can not meet this demand, then you will need to produce to prepare your experiment code, or build any of your experiment code does not the new production features. As part of it, you need to consider the following points:
• Production quenching (error handling)
• Modular for reuse
• connector call, to retrieve and store data between the production location identified in step 3.
• You'll save where the code (Git, etc.)?
• What should you add a tool to ensure that you can detect production problems and debugging models?
Step 5: Build a machine-learning applications
since it has been ready all code blocks, you can now build a machine learning application. This is why the construction of the pipeline is just different? In order to reliably perform in production, but also need to ensure that mechanisms orchestration pipeline, management and output, as well as other versions of the model is also in place. This includes how to update when a new model is in the pipeline, as well as how to improve the plumbing code after the new code into production.
If you are making the production of ML running, you can configure the ML application runtime, and connect it to the code, and other components that you created in step 1 to 4. Figure 2 shows an example of displaying a ML application generated by the runtime MCenter ParallelM.
A sample application ML: 2 FIG.
Step 6: machine learning applications deployed to the production environment
once you have a machine learning application, you can deploy up! To deploy, you need to start the ML application (or pipe) and connect them to your business applications. For example, if you use REST, ML your application will create a REST endpoint when you start your business application can call it any prediction (see Figure 3).
Figure 3: ML application to generate a REST service for business applications use
Please note that the deployment can be considered MVP "done", but this is not the end of your journey. A successful machine learning service will run for months or years, during which the need to manage, maintain and monitor.
According to the solution you selected in step 5, the deployment can be automated, it can also be manual. Tool provides automatic deployment MLOps runtime. If you do not have these tools when you run the ML application, you may need to write scripts and other software to help you deploy and manage the pipeline. You may also need to work with your IT organization to accomplish this task.
Step 7: better
recall, in step 3, you may choose a short-term position to run the production ML MVP. After deployment MVP steps 5 and 6, you may require further steps to check the results of the MVP, and to reconsider the critical infrastructure decisions. Now the code / MVP at least in the first test and run the infrastructure, you can compare and contrast different infrastructures, to see whether the need to improve.
Step 8: continued optimization
Note that steps 3 - 7 repeated in the life cycle of this machine learning application services business use cases. ML application itself can be redefined, return, transfer to the new infrastructure, and so on. You can see how MVP is being used, what feedback you get from your business, and correspondingly improved.
MLOps what?
MLOps is a comprehensive practical deployment in a production environment and management model. The above steps show you how to get started MLOps by the first model deployment. Once you have taken the above steps, you will have at least one machine learning applications in production, then you will need to manage its life cycle. Then, you may need to consider other aspects of ML lifecycle management, such as management governance model, your business comply with any regulatory requirements, kpi developed to assess the benefits of ML model for business applications brings, and so on.
Second, artificial intelligence 7 operating trends and AI and machine learning how to collaborate
The upper half, we describe the process of machine learning (ML) test for production deployment. The lower half outline helps users simplify and expand the entire machine learning Artificial Intelligence industry trends seven life cycle. Here we will describe each trend, discuss why it is important to machine learning, and when the company decided to take advantage of the trend to accelerate or improve its operating ML practice, what factors should be considered its operation.
Machine Learning (ML) life cycle
Figure 1 shows a typical machine learning (ML) life cycle. Over time, ML function with respect to the business needs to be further optimized, this cycle will repeat.
Trend: data market
many machines first challenge study plan is to find an acceptable set of data. Market data trying to solve the shortage of data collection, particularly in key areas such as health care and things, by providing a: individuals can share their data, companies can use the data and analysis of AI Artificial Intelligence platform. Market platform to ensure the security, privacy, and provides an economic model to motivate the participants.
Other market data can provide a wealth of data is difficult to obtain, but the market can provide data source and those that follow after management data and ensure the quality of the information needed.
Trends: Integrated data services
to address another point datasets shortage of synthetic datasets market. Advances in machine learning technology has proven machine learning itself can generate real data sets to train other ML algorithms, especially in deep learning space. Synthetic data is widely credited for its potential, because in comparison to large organizations have access to large data sets, artificial intelligence AI can provide a level playing field for smaller companies. Data can be anonymous synthetic version of real data sets, it can be real data samples generated by the expanded data set can also be simulated environment, such as training for autonomous vehicles in a virtual environment.
Trend three: Label service
good data sets are scarce, labeled good data sets more scarce. To solve this problem, there has been a data label market, which often focus on specific data types (such as objects in the image). Some labels from across geographical areas through coordination and coordinated management software manual labeling persons. The company is being innovative in this field, artificial and tag-based machine learning together, this is a purely human tendency has the potential to reduce the cost of the tag. Other innovations in this area include enabling enterprises to interact directly with the service provider and identity service.
Trend four: automated machine learning models
Once the appropriate data sets to find and label the next challenge is to find a good algorithm and a model train. Automated machine learning (AutoML) technology enables the algorithm / model selection and tuning process automation, get a set of input data, running a large number of training algorithms and ultra-parameter option to select the recommended final model deployment. Associated with AutoML (and often provided inside), characterized in the use of automation engineering depth characteristics as synthetic synthesis technology. AutoML software sometimes be executed in offset detection of the input data set. Some automated solution is SaaS products, while others are downloadable software that can run in the form of a container in the cloud or internal environment.
Trend five: prefabricated containers
for those who may be developing its own model, the container is a well-designed patterns of production deployment, because they make any training or reasoning can code in well-defined portable and scalable environment run. Kubernetes and other tools to further support the preparation of container-based machine learning ML scalability and flexibility. However, the assembly of the container can be a challenging task, because of the need to resolve the dependency, and the entire stack to tune and configure. Pre-built container market to solve this problem by providing the necessary libraries pre-installed and configured for container pre-configured, particularly for complex environments, such as GPUs.
Trend six: the market model
if you do not want to build their own models or training, there is the model market. Market model allows customers to purchase pre-built algorithms, and sometimes can also purchase model trained. For these cases it is useful to use the following:
(A) common use cases is sufficient, it is not necessary to customize the model training, the training does not need / reason code is customized equipment to the vessel;
(B) such as the transfer mechanism can be used to learn extend and customize the basic model;
(c) the user does not have enough training data to build their own model.
In the market model, data processing and training a good model for such an important job can be unloaded, allowing users to focus on other aspects of the operation. In other words, a key challenge is to filter content market model, in order to find the assets for your needs.
Trend Seven: application service-level artificial intelligence
Finally, for common use cases exist across business, application-level artificial intelligence AI service can eliminate the need to learn ML life cycle of the entire operation of the machine. People can subscribe to Terminal Services to perform the task of artificial intelligence, instead of creating models, training and deploying them. AI Artificial Intelligence application-level services including vision, video analytics, natural language processing (NLP), form processing, natural language translation, speech recognition, chat robots and other tasks.
Benefits and Cautions
All of these trends enables users to simplify or accelerate one or more of the various stages of operation of the machine learning ML life cycle, by offloading reuse pre-built item, or by automated particular stage. Taking into account the iterative process ML machine learning is how to achieve (for example, training usually includes tens to hundreds of experiments), automate these processes can produce more traceable, reproducible and manageable workflow. Outsourcing these tasks even easier, especially in the case of strengthening the model and algorithm (in addition to your own environment, has been tested in many environments too) can be used for basic tasks.
In other words, before using these services in your environment, there are several factors to consider:
1: Consider the applicability
Not all trends are applicable to all use cases. The most universal trend is AutoML, its wide range of applications. Similarly, the model has a very wide range of market models and algorithms available. Synthetic data sets and data marts tend to use specific classes of embodiment, the pre-built vessel may be specific to different hardware configurations (e.g., GPUs), which in turn hardware configuration for a particular purpose. Many data service label also has a specific purpose (such as image classification and form reading), but a number of consulting firms do offer custom label service. Finally, AI Artificial Intelligence-end services to very specific use cases.
2: Artificial Intelligence trust
as more are deployed ML, universal human fear of the black box performance artificial intelligence system for the trust concerns and increase the intensity of supervision. In order to benefit from AI Artificial Intelligence, companies should not only consider the mechanism of production of machine learning ML, but also consider management concerns any customer community. If not addressed, these concerns might churn, corporate embarrassment, loss of brand value or legal risks in specific.
Trust is a complex and wide range of topics, but the core is the need to understand and interpret machine learning ML, and ML sure to run properly within the expected range of parameters, from malicious ***. In particular, the decisions made by the production of ML should be interpreted - that must provide a convincing explanation. This becomes such as interpretation of provisions of other regulations in GDPR increasingly necessary. Interpretability is closely related to fairness - need to be convinced Artificial Intelligence AI is not accidentally or deliberately make biased decisions. For example, Amazon (Amazon) Rekognition AI Artificial Intelligence service also attracted attention because of prejudice.
Or to a third party automation systems, hence the need for additional understanding at each stage Since almost all the trends mentioned above are related to machine learning ML certain aspects of the life cycle of unloading, or "outsourcing" to ensure that the final production life cycle to deliver the core principles of trust. This includes understanding the algorithm deployed, whether they are used for training data sets without prejudice, and so on. These requirements will not change the life cycle itself, but requires extra effort to ensure proper follow tracking, configuration tracking and diagnostic reports.
Consider 3: diagnostic and operational management
components regardless of machine learning ML life cycle where it comes from, your business will be responsible for managing and maintaining the health of ML service in its life cycle (in addition to trends in artificial intelligence 7 fully outsourced outside services).
If so, scientists and engineers must understand the data model being deployed for training model data set and the expected safe operating parameters of these models. Since many services and markets are nascent, so there is no standardization. Users are responsible for understanding the services they use, and to ensure that services can be adequately managed with the rest of the life cycle.
（编译自：How to Move from Experimentation to Building Production Machine Learning Applications 、7 Artificial Intelligence Trends and How They Work With Operational Machine Learning，作者： Nisha Talagala）
To learn more, visit the official website Yihai software http://www.frensworkz.com/