[Recommended] Learning to Rank system

Learning to Rank

In some scenarios recommended, not only to consider the issue forecast, but also consider scheduling problem. For example, in Query scenario, the user selects a desired item so that further forward in sorting results. For example, I use Baidu search for one thing, I will be able to find me what I want on the first page, so in this case would be far better than I would also like the ability to turn one can find the results I want.

Ranking Model:

Ranking based on the importance of showing a structure based solely between pages to determine the importance of the document, such as PageRank, TrustRank and so on.

Ranking based on the degree of correlation, since fewer factors to consider PageRank conventional, generally considering the correlation with the content of the document, an inverse document frequency, length of the document, and several factors, but the actual sorting process to be considered more features, such as the number of words pageRank value pages, and documents that match the query, the page URL link address length and so have an impact on page rank. Machine learning search engine can provide large amounts of data need, therefore Ranking converted into a supervised machine learning.

 

 

LTR of three strategies:

Pointwise

An idea is the degree of correlation between the query and the doc as a label, such as label third gear, the problem becomes a multi-class classification problem, the model has McRank, svm, maximum entropy. Another idea is that the degree of correlation between the query and the doc as a fraction using the regression model, the regression model has a classic linear regression, dnn, mart and so on.

Pairwise

Pairwise method is composed of a document on the document, not only consider a single document, but the document group to consider whether it is reasonable, for example, three query 1 returned document doc1, doc2, doc3, there are three groups of ways, <doc1, doc2> , <doc2, doc3>, <doc1, doc3>. When the three documents as 3,4,2 original label, after three examples of this group to have a new score (expression of this order is reasonable), can use this data to classify learning, models such as SVM Rank, there RankNet (C. Burges, et al. ICML 2005), FRank (M.Tsai, T.Liu, et al. SIGIR 2007), RankBoost (Y. Freund, et al. JMLR 2003).

One problem is that doc Pairwise discrimination on different score is the same, but more inclined to click on the forefront of information systems which result in the user, it should focus on the most relevant top surface. And the range of available models is not to sort performance as the ultimate goal, will lead to the final ranking results are unsatisfactory.

Listwise

Pointwise the training set in each document as a training example, pairwise method with any of the search results of a query in two as a training document for example, Listwise method is to put all the search results list corresponding to each query as a whole as a training examples,

ListNet common model, such as the KL distance (cross-entropy) distribution arrangement between the predicted probability of correct sequencing and sorting as a loss function, LambdaMART frame is actually MART, the main innovation is intermediate calculation using gradient Lambda, similar to IR NDCG evaluation in this derivation can not be converted into a derivative of the function and the physical meaning rich gradient.

Policy description:

Pointwise：

Advantage of this method is simple, in that lack (relative order between doc) between the position information of the sample is not considered

The processing object is a document, the document will be converted into feature vectors, machine learning systems learning from document to document classification or regression function scoring, scoring sorting result is the search results. The figure is the training set of manual annotation:

 

 

Which, Label is a correlation, feature is Cosme query and document similarity score, PageRank value and the value of the query terms Proximity page.

For a machine learning system, according to the training data, a need for a linear function of scoring: Score (Q, D) = A * + B * the PM the CS * PR + C + D

If the score is greater than the set threshold, it may be considered relevant, if less than the set threshold value may be deemed irrelevant. By training examples, you can obtain an optimal a, b, c, d parameter combination, when determining these parameters, machine learning systems, even if the learning completion. After scoring function to use this correlation is determined.

Available models:

LR，GBDT+lr

  

Listwise the scheduling problem into a binary classification problem.

As shown, for the following query Q1, manual labeling, the highest score Doc2 = 5, followed Doc3 4 minutes, the worst is Doc1 3 minutes, after this has turned relative relationship: Doc2> Doc1, Doc2> Doc3, Doc3> Doc1, so the problem can be sorted naturally into two documents to determine any relationship, and any judge would order two documents called a very familiar classification.

 

Reference Ranking SVM Learning to Rank's profile gives a good example to explain this problem: Given a query q, the document d1> d2> d3 (i.e. document than the document d1 d2 related documents relevant than documents d2 d3, x1 , x2, x3 are the feature d1, d2, d3) of. In order to sort the use of machine learning, we will sort into a classification problem. We define a new training samples, so x1-x2, x1-x3, x2-x3 is positive samples, make the x2-x1, x3-x1, x3-x2 is negative samples and then training a two classifiers (SVM) to classify these new training samples, as shown below:

FIG Each ellipse represents a query point in the ellipse representative of those documents, triangles represent and to calculate the degree of correlation is very relevant to the query, circles represent a general correlation, cross number represents uncorrelated. We convert a single document left to right in the document to (di, dj), solid squares represent positive samples, that di> dj, hollow squares represent negative samples, ie, di <dj. After sorting [] problem into a classification problem, we can use a common machine learning methods to solve the problem. The main algorithms: SVM Rank, RankBoost (2003), RankNet (2007)

 

Listwise

Column method according to documents obtained K training examples of training the best scoring function F, for a new query function F scored for each document, and then sorted in descending order of score, is the corresponding search results.

 

 

So the key question is: get the training data, training on how to get the best scoring function? We know that for search engines, a user enters a query Q, the search engine returns search results, we assume that the search result set containing A. B and C 3 documents, the search engine to sort the search results, and that three documents There are six permutations order mode: ABC, ACB, BAG, BCA , CAB and CBA and each one of the possible permutations of the search results are sorted. For a scoring function F, the correlation of the three scoring search result document, to give three different relevance score F (A), F (B ) and F (C), according to the three scores can be calculating combinations of six kinds of arrangement of the respective probability value. Different scoring functions which six kinds of permutations and combinations of search results probability distribution is not the same. Here describes a training method, which is the probability that the search results based on the distribution of permutations and combinations to train, and in this way the figure is diagrammatic schematic training process.

The figure shows a specific example of training that queries Q1 and corresponding scores three documents, this score is made up of artificial playing, it can be seen as the standard answer. It is contemplated that there is an optimal scoring function g, the query Q1, its scoring result be: A Document 6 points, B document 4 points, C document 3 points.

Our task is to find a function that the function of the search results Q1 scoring sequence and manual scoring sequence as similar as possible.

  Reference article:
https://www.jianshu.com/p/ff9853a4892e
https://www.cnblogs.com/Lee-yl/p/11200535.html