1. Concept

Compared with domain adaptation , the most significant difference of domain generalization is that the test set cannot be accessed during training .

The loss function of domain generalization can generally be described as the following form:

$\epsilon ^t\leq \sum\pi^*\epsilon^i(h)+\frac{\gamma +\rho }{2}+\lambda_H,(P^t_X,P^*_X)$

This formula is divided into three items: the first item $\sum\pi^*\epsilon^i(h)$ represents the linear combination of the weights of each training set , where π is the coefficient that minimizes this item; the second item $\frac{\gamma +\rho }{2}$ represents the inter-domain distance , which $\gamma$ represents the minimum distance between the target domain and the source domain, $\rho$ Indicates the maximum distance between pairwise combinations of source domains; the third item $\lambda_H,(P^t_X,P^*_X)$ indicates ideal joint risk, which can be ignored under normal circumstances .

2. Classification

1. Data manipulation

This method is embodied in the operation of the data set, which is mainly divided into data augmentation (Data augmentation) and data generation (Data generation)

Among them, the main way of data enhancement is to adjust the size, color, brightness and contrast of the image, rotate it, add noise and other operations. It can be divided into: relevant data enhancement and confrontational data enhancement according to the direction of its enhancement .

There are three main ways of data generation : VAE, GAN (against generation), Mixup (mixed enhancement), the main purpose is to enhance the generalization ability of the model.

2. Representation learning

The method can be characterized as:

There are four main methods to characterize the characteristics of the domain by learning each part of the above formula

①Kernel-based method : traditional method, mainly relying on nuclear projection techniques

②Domain adversarial learning : confrontation method, based on confrontation network for confusion

③Explicit feature alignment : Explicitly reduce the difference between domains, domain alignment

④Invariant risk minimization : paradigm method

⑤Feature disentanglement : Decoupling, extracting common features in the same category

Mainly divided into two types: 1. UndoBias : Divide the weight into two types $w_i=w_0+\Delta_i$ (including $w_0$ the common features of all domains and $\Delta_i$ the private features of each domain)

2. Generative modeling : Decoupling using a generative network

3. Learning strategy

①Meta-learning (source learning)

Decompose the source domain into several small tasks

②Ensemble learning (integrated learning)

It is considered that the target domain is a linear combination of source domains, and in actual operation, various results are combined according to certain weights (similar to voting)

[Transfer Learning] Domain Generalization