2021-02-21 16:43:00
Xiao Xiao from the concave temple
Qubit report | public account QbitAI
Good news, Google has open sourced the AutoML algorithm library!
This platform called "Model Search" can not only use multiple AutoML algorithms to automatically write the AI model you want, but also help you choose the best written one.
The most important thing is that it can be used in all fields .
In other words, the AutoML algorithm that used to only support single-domain model search such as NLP and image classification is now integrated on a platform that can help you build any AI model.
Now, without redesigning parameters or repeated fine-tuning, "AI designer" can help you write the model you want.
What platform is "model search"
Previously, the AutoML algorithm has been applied to various fields to reduce the burden of neural network design experts.
The purpose of this algorithm is to allow AI to design a neural network and automatically make a reasonable combination of factors such as network depth, layer type, structure, and optimization algorithm. The effect is usually better than manual direct design.
However, this method of designing AI models by AI will face two problems.
For one, these algorithms usually only target a specific field and cannot be applied to other fields.
For example, the AutoML algorithm for NLP cannot design an AI model for image classification.
Second, the amount of calculation is huge .
The previous NAS and PNAS algorithms often need to train thousands of models to find the best effect.
In response to these two problems, Google has now launched a "model search" open source platform, dedicated to solving them.
This system consists of multiple trainers, a search algorithm, a migration learning algorithm, and a database containing multiple evaluation models.
In the process, each trainer will independently build models and conduct experiments, but these trainers can share data and use horizontal search to decide what model to try next.
"Model search" can build a neural network model based on a set of predefined modules. Each module contains a classic microstructure, including some layers in LSTM, ResNet, or Transformer, and so on.
This micro-structured model also reduces the scale of the search , because it explores the structure of these models rather than the more detailed basic parts.
In order to further improve efficiency and accuracy, this algorithm can also perform transfer learning when the trainer completes various experiments . Mainly through two methods: knowledge extraction and parameter assignment.
Through knowledge extraction , the new model can learn the loss function from the high-performance model to improve its own accuracy; and through parameter allocation , the new model adopts some of the parameters in the previous training model and initializes the remaining parameters to train faster.
In the process of gradual iteration, the best model is "searched".
Google said that "model search" is an algorithm that is adaptive, greedy, and converges faster than reinforcement learning algorithms.
This algorithm currently has the following functions:
- You can run multiple AutoML algorithms on the data, automatically search for suitable model structures, model fusion methods, and select the best model.
- It is possible to compare different models found during the search;
- You can design a special neural network layer and apply it yourself.
Currently, "Model Search" supports the Tensorflow framework.
In other words, each module can implement any function that takes a tensor as input.
Designed better than humans, more efficient than PNAS
After experiments, the AI model produced by the "model search" platform is indeed not bad.
The authors used the "model search" platform to try to write a voice AI model, whose main function is keyword detection and language recognition.
In the figure below, the solid line is the model iteration accuracy written by AI, and the dashed line is the SOTA model manually designed before.
Obviously, whether it is the minimum number of iterations or the final iteration accuracy, the models written by AI on the "model search" platform are much better than those designed manually.
In other words, the number of parameters used in the design is less (compared to the 315,000 manually designed, AI only needs 184,000), and the accuracy has increased.
So, the search effect of this "model search" framework is better than other search algorithms that use AI to write AI models?
The authors tried an image classification model with the CIFAR-10 data set.
The test found that after trying to write 209 models with AutoML , the best model has reached 91.83% accuracy.
Previously, NasNet needed 5807 attempts and PNAS needed 1160 attempts to achieve the same accuracy.
In other words, the AI model designed with this platform can not only achieve better results than human design in some areas, but also faster than other "AI designers".
If you don't want to work hard to adjust the parameters, this is definitely a very ideal model design platform.
Does not include all AutoML algorithms
So, Google really open sourced its previous fee-based projects?
No no no.
More advanced AutoML algorithms are currently charged.
This is a project called AutoML Tables. It can help you automatically build and deploy the most advanced machine learning models without writing code.
The AutoML algorithm with the best performance is currently included in it.
At present, Google has integrated AutoML, MLOps, and AI Platform into a larger AI Platform platform.
Of course, there is also a fee.
In other words, the current open source "model search" platform only contains part of the AutoML algorithm.
about the author
Hanna Mazzawi, Google research engineer, research direction is machine learning, algorithm design and analysis, mathematical software.
Xavi Gonzalvo, both masters and Ph.Ds, graduated from Ramon Llull University in Spain. He is currently a research scientist at Google, engaged in machine intelligence related work.
If you want to quickly write the AI model you need, you can get started with this project~
Project address:
https://github.com/google/model_search
Reference link:
https://cloud.google.com/automl-tables
https://ai.googleblog.com/2021/02/introducing-model-search-open-source.html