Summary of various machine learning methods/learning paradigms

Summary of various machine learning methods (learning paradigms)

reinforcement learning

Getting Started

"Introduction to the Principles of Reinforcement Learning in Simple Strategies". Edited by Guo Xian et al. Electronic Industry Press

An Introduction to Reinforcement Learning, Sutton and Barto, 1998

Algorithms for Reinforcement Learning, Szepesvari, 2009

Features

1. No supervised data, only reward signal
2. Reward signals are not necessarily real-time, but are likely to be delayed, sometimes significantly delayed
3. Time (series) is an important factor
4. Current behavior affects subsequent data received

Dual Learning

In their paper submitted to NIPS 2016, Dr. Qin Tao of Microsoft Research Asia proposed a new paradigm of machine learning—dual learning.

Getting Started

Qin Tao, Microsoft Research Asia: The Beauty of Symmetry in Dual Learning | Hard Innovation Open Class Summary. https://zhuanlan.zhihu.com/p/27513847

Dual Learning: A New Machine Learning Paradigm that Reduces Data Labeling Costs from $20M to $2M. Xinzhiyuan. http://www.sohu.com/a/121198568_473283

Di He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, and Wei-Ying Ma, Dual Learning for Machine Translation , NIPS 2016. ( https://papers.nips.cc/paper/6469- dual-learning-for-machine-translation )

solved problem

Reduce reliance on large-scale annotated data

The great success of deep learning is due to the large scale of labeled data. However, there are two limitations: 1. Manual annotation is expensive to obtain labels; 2. Large-scale annotation data cannot be collected in many tasks, such as medical treatment or mutual translation between small languages.

core idea

Many AI applications involve two dual tasks, such as translation from Chinese to English and translation from English to Chinese in machine translation, dual duality of speech recognition and speech synthesis in speech processing, and duality in image understanding. Image-generated text and text-based image generation are duals, answering questions and generating questions in question answering systems are duals, and searching for relevant web pages for search terms in search engines and generating keywords for web pages are duals. These dual AI tasks can form a closed loop, enabling learning from unlabeled data.

Will have a big impact on the field of machine learning

First, many deep learning researchers believe that the next breakthrough in AI and deep learning is learning from unlabeled data.

Second, reinforcement learning has had limited success in complex practical applications. And dual learning provides a way to capture reward information for reinforcement learning and confirms the possibility of reinforcement learning being successful in complex applications such as translation.

transfer learning

Getting Started

Wang Jindong. A Concise Manual of Transfer Learning. 2018

authoritative scholar

Professor Yang Qiang, Hong Kong University of Science and Technology

solved problem

1. Insufficient labeled data
2. Insufficient computing power
3. Personalized requirements: further improve the generalization ability of the model
4. Application-specific requirements, such as cold-start problems for recommender systems

federated learning

Getting Started

"Google Research | Federated Learning: Collaborative Machine Learning Without Centralized Storage of Training Data". WeChat public account: Google Developer

solved problem

Standard machine learning methods require training data to be centralized on a single machine or in a data center. In order to process this data and improve services, a secure and robust cloud infrastructure needs to be built.

Through federated learning, mobile phones can collaboratively learn a shared predictive model while keeping all training data on-device, enabling machine learning without storing the data in the cloud. At the same time, by introducing model training to the device, it goes beyond the previous model of using local models to predict mobile devices.

working principle

Your device downloads the current model, improves it by learning from the data in your phone, and summarizes the changes in small updates. Only updates to this model are sent to the cloud via encrypted communication, where they are immediately averaged with other user updates to improve the shared model. All training data remains on your device and no individual user updates are stored in the cloud.

Advantage

Build smarter models, reduce latency, and reduce power consumption while ensuring privacy. In addition to providing updates for shared models, instant access to improved models from your phone provides a personalized experience based on how you use your phone.