Cosine similarity formula derivation and code implementation

1. Why use cosine similarity?

• Definition of similarity between two points on the spatial dimension
  • Between two points in the space dimension there isangle and direction, the angle range is [0,180]
  • The more similar two points are ,directioncloser to the same direction ,AngleThe smaller it should be , the closer it is to 0 degrees ;
  • The two dots have opposite meanings ,directionThe closer to the opposite direction ,AngleIt should be larger and closer to 180 degrees ;
  • The two points are not similar ,directionvertical ,Angleshould be close to 90 degrees ;
• Why use cosine instead of sine ?
  • According to the similar definition above, we need to find the angle range in [0,180]monotonicityThe function
  • As shown in the figure below, the range of the included angle of the sine value is [0,180] , which is not a monotonic value [does not meet]
  • The range of the cosine value included angle is [0,180] , which is monotonic
    • Angleis 0 , it should be more similar ,Angle valuebigger (1)
    • Angleis 90 , there should be no correlation ,Angle valueis 0
    • Angleis 180 , it should mean the opposite ,Angle valuesmaller (-1)
      
• Summarize
  • Cosine monotonicity conforms to our relationship in the spatial dimension position, so use cosine similarity 【Self-understanding】

2. Formula derivation

2.1 Derivation of trigonometric function cosine formula

$$\begin{array}{rl}cos(\theta & =\frac{{AB}^2+{AC}^2-{BC}^2}{2\ast AB\ast AC}\end{array}$$

• Displacement
  $$\begin{array}{r}cos\left(\theta \right)=\frac{AD}{AC}\end{array}$$
• According to the principle of Pythagorean theorem,
  $$\begin{array}{r}{AC}^2={AD}^2+{CD}^2(1)\\ {BC}^2={BD}^2+{CD}^2(2)\end{array}$$
• 公式1-公式2
  $$\begin{array}{rl}{AC}^2-{BC}^2& ={AD}^2-{BD}^2(3)\end{array}$$
• $AB,AD,$Relationship between$B$$D$$AD$ isan unknown value(butthe cosine formula requires parameters),$AB$ isa known value, so get$BD$ related formula, put$BD$消掉
  $$\begin{array}{rl}AB& =AD+BD\\ BD& =AB-AD(4)\end{array}$$
• 公式4带入公式3
  $$\begin{array}{rl}{AC}^2-{BC}^2& ={AD}^2-{(AB-AD)}^2\\ {AC}^2-{BC}^2& ={AD}^2-{AB}^2+2\ast AB\ast AD-{AD}^2\\ {AC}^2-{BC}^2& =-{AB}^2+2\ast AB\ast AD\\ AD& =\frac{{AB}^2+{AC}^2-{BC}^2}{2\ast AB}(5)\end{array}$$
• 5- dimensional coefficient
  $$\begin{array}{rl}cos\left(\theta \right)& =\frac{AD}{AC}\\ & =\frac{\frac{{AB}^2+{AC}^2-{BC}^2}{2\ast AB}}{AC}\\ & =\frac{{AB}^2+{AC}^2-{BC}^2}{2\ast AB\ast AC}\end{array}$$

2.2 Derivation of trigonometric function vector cosine formula

$$\begin{array}{rl}cos(\theta & =\frac{\vec{a}\ast \vec{b}}{\parallel a\Vert \ast \Vert b\Vert }\end{array}$$

• 向量公式
  $$\begin{array}{rl}\vec{c}& =\vec{a}-\vec{b}(1)\\ AC& =\parallel \vec{b}\Vert (2)\\ AB& =\Vert \vec{a}\Vert (3)\\ BC& =\Vert \vec{c}\Vert (4)\end{array}$$
  
• $Form1,2,3,4$ ,the infinitive
  $$\begin{array}{rl}cos\left(\theta \right)& =\frac{{AB}^2+{AC}^2-{BC}^2}{2\ast AB\ast AC}\\ cos\left(\theta \right)& =\frac{{\Vert \vec{a}\Vert }^2+{\Vert \vec{b}\Vert }^2-({\parallel \vec{a}-\vec{b}\parallel )}^2}{2\ast \parallel \vec{a}\Vert \ast \Vert \vec{b}\Vert }\\ cos\left(\theta \right)& =\frac{{\Vert \vec{a}\Vert }^2+{\Vert \vec{b}\Vert }^2-{\Vert \vec{a}\Vert }^2+2\ast \vec{a}\ast \vec{b}-{\Vert \vec{b}\Vert }^2}{2\ast \Vert \vec{a}\Vert \ast \Vert \vec{b}\Vert }\\ cos(\theta & =\frac{\vec{a}\ast \vec{b}}{\Vert a\Vert \ast \Vert b\Vert }\end{array}$$

3. Cosine similarity code implementation

• The code is from the book:Advanced Deep Learning: Natural Language Processing

  import numpy as np
  
  def preprocess(text):
      """
         语料库预处理
  
         :param text:句子字符串
         :return:
              corpus 是单词ID 列表
              word_to_id：是单词到单词 ID 的字典
              id_to_word 是单词 ID 到单词的字典
      """
      text = text.lower().replace('.', ' .') # 单词全为小写
      words = text.split(' ') # 以空格分隔
      word_to_id = {
        
        }
      id_to_word = {
        
        }
      for word in words:
       if word not in word_to_id:
           new_id = len(word_to_id)
           word_to_id[word] = new_id
           id_to_word[new_id] = word
      corpus = np.array([word_to_id[w] for w in words])
      return corpus, word_to_id, id_to_word
  def create_co_matrix(corpus, vocab_size, window_size=1):
      """
      语料库生成共现矩阵
  
         :param corpus:corpus 是单词 ID 列表
         :param vocab_size:词汇个数
         :param window_size:窗口大小
         :return:
              共现矩阵
      """
      corpus_size = len(corpus)
      co_matrix = np.zeros((vocab_size, vocab_size), dtype=np.int32)
      for idx, word_id in enumerate(corpus):
          for i in range(1, window_size + 1):
              left_idx = idx - i
              right_idx = idx + i
              if left_idx >= 0:
                  left_word_id = corpus[left_idx]
                  co_matrix[word_id, left_word_id] += 1
              if right_idx < corpus_size:
                  right_word_id = corpus[right_idx]
                  co_matrix[word_id, right_word_id] += 1
      return co_matrix
  def cos_similarity(x, y, eps=1e-8):
      """
      余弦相似度函数
  
      :param x:x坐标值
      :param y:y坐标值
      :param eps:默认值为1e-8，防止分母为0
      :return:
          余弦相似度值
      """
      nx = x / (np.sqrt(np.sum(x ** 2)) + eps)
      ny = y / (np.sqrt(np.sum(y ** 2)) + eps)
      return np.dot(nx, ny)
  text = 'I say hello and You say goodbye.'
  corpus, word_to_id, id_to_word = preprocess(text)
  print("corpus为：",corpus)
  print("word_to_id为：",word_to_id)
  print("id_to_word为：",id_to_word)
  vocab_size=len(set(corpus))
  C=create_co_matrix(corpus, vocab_size, window_size=1)
  print("共现矩阵为：",C)
  c0 = C[word_to_id['you']] # you的单词向量
  c1 = C[word_to_id['i']] # i的单词向量
  print('you和i的相似度为',cos_similarity(c0, c1))