"ROTATE: KNOWLEDGE GRAPH EMBEDDING BY RELATIONAL ROTATION IN COMPLEX SPACE" paper reading

Published at ICLR 2019.

abstract

Research on the task of learning representations of entities and relations in knowledge graphs to predict missing links. The success of this task depends heavily on the schema for modeling and inferring relationships. In this paper, we propose RotatE, a novel approach to knowledge embedding graphs, which can model and infer various relational schemas: symmetric/asymmetric, inversion, composition. RotatE models each relation as a rotation from a source entity to a target entity in a complex vector space. In addition, we also propose a new self-adversarial negative sampling technique for the effectiveness training of the RotatE model. Experiments prove that RotatE is not only scalable, but also can infer and model a variety of relationship patterns, and can surpass the existing SOTA methods in link prediction patterns.

1.introduction

Finding ways to model and infer patterns (e.g., symmetry/asymmetry, inversion, composition) from observed facts is important in predicting missing links. However, existing methods cannot model all of the above patterns, so we find ways to model and infer the above three types of relational patterns.

Determine the motivational value of the Euler identity: $e^{i\theta }=cos\theta +isin\theta$ , which shows that a complex number can be viewed as a rotation of the complex space. Given a triplet$(h,r,t)$ , we expect$t=h\circ r$，其中$h,r,t\in {\mathbb{C}}^k$ is embeddings, and$|r_i|=1$,$\circ$ represents the product of Hadamard. For each dimension in the complex plane, we expect:

$t_i=h_i{r}_i，h_i,r_i,t_i\in \mathbb{C},|r_i|=1.$

Such a simple operation can effectively model the three relationships mentioned before: symmetric/antisymmetric, inversion, composition. Example: a relation $r$ is symmetric iff embeddingrrEach component of$r$$r_i=e^{0/i\pi }=\pm 1.$ The two relations are inverse iff and their embeddings are conjugates:$r\; 2$$r_2=\overline{r_1}$, a relation $r_3=e^{i{\theta }_3}$These two relations are $r_2=e^{i{\theta }_2},r_1=e^{i{\theta }_1}$Definition iff $r_3=r_1\circ r_2(i_3=i_1+i_2)$ . Furthermore, RotatE can be extended to large knowledge graphs because it is almost linear in time and space.

In order to effectively optimize RotatE, we further propose a self-adversarial negative sampling method to generate negative samples according to the current entities and relations. This approach is very general and can be used in many existing knowledge graph embedding methods. Experiments prove that RotatE achieves SOTA.

2.related work

3.RotatE：relational rotation in complex vector space

3.1 modeling and inferring relation patterns

There are three very important relationship patterns in the knowledge graph: summer, inversion and composition. Give their definitions: \
Definition 1: A relation $r$ is symmetric/antisymmetric, if for$\forall x,y:$

$r(x,y)\Rightarrow r(y,x)(r(x,y)\Rightarrow \neg r(y,x))$

Definition 2: A relation $r_1$is the relation $r_2$The inverse, if for $\forall x,y:$

$r_2(x,y)\Rightarrow r_1(x,y)$

Definition 3: A relation $r_1$From $r_2,r_3$Formation, if for $\forall x,y,z:$

$r_2(x,y)\bigcap r_3(y,z)\Rightarrow r_1(x,z)$

3.2 modeling relations as rotations in complex vector space

Given a triplet $(h,r,t)$：

$t=h\circ r，where|r_i|=1$

For each element in the embedding, we have $t_i=h_i{r}_i$，且$h,t\in {\mathbb{C}}^k$，$r_i\in \mathbb{C},|r_i|=1$ , cause this$r_i$specific form $e^{i{\theta }_{r,i}}$, is θ r , i \theta_{r,i} about the origin in the complex plane$i_{r,i}$A counterclockwise rotation of , only affects entity embeddings in negative vector spaces. We call the proposed model RotatE due to its rotation property. According to the above definition, for each triplet $(h,r,t)$ , we define the distance function of RotatE as follows:

$d_r(h,t)=||h\circ r-t||$

By defining each relation as a rotation in a complex vector space, RotatE is the only model that can model and infer the above three relation patterns.

3.3 optimization

We use a loss function similar to the negative sampling loss to efficiently optimize distance-based models:

$L=-logs(c-d_r(h,t))-\sum _{i=1}^n\frac{1}{k}logs(d_r(h_i^{\prime },t_i^{\prime })-c),$

where $\gamma$ is a fixed margin,$(h_i^{\prime },r,t_i^{\prime })$是i-th negative triplet。

We also propose a new method to generate negative samples. The negative sampling loss uses a uniform method to sample negative triplets. Such a uniform negative sampling method has the problem of inefficiency (because many samples are obviously wrong as the training progresses, and cannot provide any meaningful information). Therefore we propose a negative sampling called self-adversarial, which samples negative triples according to the current embedding model. Specifically, we sample negative triples from the following distribution:

the final form of the negative sampling loss with self-adversarial training:

4.experiments