In transfer learning, marginal probability distribution and conditional probability distribution are two important concepts. Below I will elaborate on these two concepts from my own understanding.
marginal probability distribution
Marginal Probability Distribution, or P(X), refers to the characteristic distribution of data in transfer learning. Different P(x) means that the data generation mechanisms are different, such as images from different angles, lighting or backgrounds, physiological data and behavioral data generated by different subjects. They all obey different feature distributions, that is, the marginal probability distributions are inconsistent. In transfer learning, as long as the source domain and target domain data obey different distribution laws, we believe that there is a difference in marginal probability distribution between the two. This difference is called distribution shift or domain shift.
conditional probability distribution
Conditional Probability Distribution (Conditional Probability Distribution), namely P(y|x), in transfer learning refers to the distribution of labels under the conditions of given features. Ideally, the conditional probability distributions of the source and target domains are the same. However, this does not always hold true in reality. For example, the physiological data of different subjects are usually different; and even if the same subject performs the same action in different states or times, the physiological data will also be different.
In addition, in transfer learning, it is very important to understand the difference between marginal probability distribution and conditional probability distribution, because this can help us choose an appropriate transfer learning strategy. For example, if the source domain and the target domain have a large difference in the marginal probability distribution, but a small difference in the conditional probability distribution, then we may choose to use feature selection or feature transformation methods for transfer learning. On the contrary, if there is a large difference in the conditional probability distribution between the source domain and the target domain, then we may need to use more complex transfer learning methods, such as re-labeling some data in the target domain, or using methods such as adversarial training to solve this problem .