The difference between CRF and MRF:
Just a follow-up on Eren's answer:
Let X denote a multi-dimensional input (i.e., the features), and let Y denote a multi-dimensional output (i.e., the structured label). Typically, we have the features x, and are interested in predicting the distribution of the label P(Y|X=x), or the best possible label argmax_y P(Y=y|X=x).
A CRF is essentially a structured extension of logistic regression, and models the CONDITIONAL probabilities P(Y|X). It does not model anything else.
A MRF models the JOINT probabilities of both Y and X simultaneously. In other words, it models P(Y,X). You can do various tricks to compute P(Y|X=x) for a given input x.
The advantage of CRFs is that they are more directly modeling the standard prediction problem P(Y|X=x). As such, they're often more accurate. But they cannot (without further modifications) be used for anything other than the standard prediction problem.
The advantage of MRFs is that they are fully generative, and so can model arbitrary prediction problems. For instance, suppose for some reason that you had missing values in your input x. Then an MRF can marginalize over the missing values because it models the full probability distribution. As another example, suppose you had the y and wanted to predict X. Then an MRF can also predict P(X|Y=y), but a CRF cannot.
For me, CRFs are generally much more useful than MRFs, because I'm generally interested in the standard prediction problem P(Y|X=x).
赵家平科学网回答：
CRF and MRF are two popular graphical models, which is an intuitive model to model all kinds of networks.
Recently deep belief network has aroused a huge surge in Machine Learning community, and it got state of the art results in numerous fields, like object recognition, acoustic understanding and topic modeling. DBN is also one kind of graphical models.
As we know, CRF is a discriminative model, while MRF is a generative model. Naive Bayes is probably the simplest MRF, while logistic regression is one kind of simple CRFs. The key difference between these two kinds of models is that: MRF is trying to model a joint distribution p(X,Y), however, CRF aims to build a conditional distribution p(Y|X). To be more clear, there are no kinds of potential functions like g(x1,x2) in CRF. Nevertheless, the learning methods & procedures are quite the same, and maximum likelihood works for both model. By the way, for parameter learning in DBN, Prof. Hinton developed a method called contrastive divergence, which is pseudo-likelihood.
One important theorem in undirected graph is:
Hammersley-Clifford theorem: a positive distribution p(y) > 0 satisfies the conditional independence properties of an undirected graph G if and only if p can be represented as a product of factors, one per maximal clique.
参考自：