Learning Algorithms for Hash Functions

foreword

If you are interested in this article, you can click " 【Visitor Must Read-Guide Page】This article includes all high-quality blogs on the homepage " to view the complete blog classification and corresponding links.

---

overview

Two steps for hash function learning:

• Convert to binary code: You can first reduce the dimension to real numbers, and then convert to binary, or you can directly learn a binary code;
• Learning hash mapping function: Design or learn hashing methods based on binary codes, so that similar elements are close and dissimilar elements are far away.

According to the properties of the hash function, it can be classified as follows:

• Data-Independent Methods (Data-Independent Methods)
  • Features: The hash function has nothing to do with the training set, usually random projection or manual construction
  • 举例：Locality-sensitive hashing (LSH), Shift invariant kernel hashing (SIKH), MinHash;
• Data-Dependent Methods
  • Features: The hash function is learned through the training set
  • Classification: unimodal hashing (unsupervised/supervised), ranking-based methods (supervised information is sorted sequence), multimodal hashing, deep hashing

---

Unsupervised Hashing

Problem definition:

• Input: feature vector { xi } \{\mathbf{x}_i\}{ xi​} , corresponding to the matrix$\mathbf{X}$；
• Output: binary code { bi } \{\mathbf{b}_i\}{ bi​} , corresponding to matrix$B$ , similar features correspond to similar codes

PCA Hashing (PCAH)

Select matrix ${\mathbf{XX}}^{\top }$ Maximum$m$ eigenvectors to form a projection matrix$\mathbf{W}\in {\mathbb{R}}^{d\times m}$ , and define the hash function as:
$$h(\mathbf{x})=\mathrm{sgn}\left({\mathbf{W}}^{\top }\mathbf{x}\right).$$

Spectral Hashing (SH)

min ⁡ { y i }     ∑ i j W i j ∥ y i − y j ∥ 2  s.t.    y i ∈ { − 1 , 1 } k ∑ i y i = 0 1 n ∑ i y i y i ⊤ = I \begin{aligned} \mathop{\min}\limits_{\left\{\mathbf{y}_i\right\}} \ \ \ & \sum_{i j} W_{i j}\left\|\mathbf{y}_i-\mathbf{y}_j\right\|^2 \\ \text { s.t.} \ \ \ & \mathbf{y}_i \in\{-1,1\}^k \\ &\sum_i \mathbf{y}_i=0 \\ &\frac{1}{n} \sum_i \mathbf{y}_i \mathbf{y}_i^\top=\mathbf{I} \end{aligned} { yi​}min​    s.t.   ​ij∑​Wij​∥yi​−yj​∥2yi​∈{ −1,1}ki∑​yi​=0n1​i∑​yi​yi⊤​=I​

W $W_{ij}$为${\mathbf{x}}_i$和$x_j$The similarity, the second constraint means that after all the data is mapped, each bit is $1$ and$The\; number\; of\; -1s$ is the same, the third constraint means that there is no correlation between different bits. The above optimization problem is an integer programming, which is difficult to solve. You can useyi ∈ { − 1 , 1 } k \mathbf{y}_i \in\{-1,1\}^kyi​∈{ −1,1}The k constraint is canceled, the problem is relaxed, and the obtained{ yi } \{\mathbf{y}_i\}{ yi​} each bit by$\mathrm{sgn}$ function mapping to get the final binary code.

Anchor Graph Hashing (AGH)

For $X$ uses k-means clustering to get{ uj ∈ R d } j = 1 m \{\mathbf{u}_j\in \mathbb{R}^d\}_{j=1}^m{ uj​∈Rd}j=1m​, redefine the matrix $\mathbf{Z}\in R^{n\times m}$：
$$Z_{ij}=\{\begin{array}{l}\frac{\exp \left(-D^2\left(x_i,u_j\right)/t\right)}{{\sum }_{j^{\prime }\in {\mathcal{J}}_i}\exp \left(-{\mathcal{D}}^2\left({\mathbf{x}}_i,{\mathbf{u}}_{j^{\prime }}\right)/t\right)},\forall j\in {\mathcal{J}}_i\\ 0,\text{ otherwise }\end{array}$$

where $J_i$is a set of subscripts, corresponding to { uj ∈ R d } j = 1 m \{\mathbf{u}_j\in \mathbb{R}^d\}_{j=1}^m{ uj​∈Rd}j=1m​middle $s$ away from${\mathbf{x}}_i$The index of the nearest point. Define the hash function as:
$$h(\mathbf{x})=\mathrm{sign}\left(W^{\top }z(\mathbf{x})\right),$$

W = n Λ − 1 / $W=\sqrt{n}{L}^{-1/2}V{\Sigma }^{-1/2}$ ,$L=\mathrm{diag}\left(Z^{\top }1\right)$，$V$ and$\Sigma$ by matrix$L^{-1/2}{Z}^{\top }{\Lambda }^{-1/2}$ of the eigenvectors and eigenvalues. The overall goal of this method is consistent with Spectral Hashing, but by introducing Anchor, the problem solving is accelerated, and the time complexity is from$O\left(n^3\right)$down to$O\left(n{m}^2\right)$. For details, please refer tothe original paper.

---

Supervised Hashing

Problem definition:

• Input: feature vector { xi } \{\mathbf{x}_i\}{ xi​} , corresponding to the matrix$X$ , category vector{ yi } \{\mathbf{y}_i\}{ yi​} , corresponding to matrix$\mathbf{Y}$
• Output: binary code { bi } \{\mathbf{b}_i\}{ bi​} , corresponding to matrix$B$ , the same category corresponds to a similar code

The common algorithms are as follows. For specific information, please refer to the reference materials, which will not be expanded here.

---

Ranking-based Methods

This part belongs to the category of supervised hashing, but the supervision information has changed from marking to sorting information, such as triples $\left(x,x^{+},x^{-}\right)$. Common algorithms covered in this section are as follows:

---

References

• Learning to Hash
• Learning to Hash for Big Data: A Tutorial