【中英】【吴恩达课后测验】Course 5 - 序列模型 - 第三周测验 - 序列模型与注意力机制

【中英】【吴恩达课后测验】Course 5 - 序列模型 - 第三周测验 - 序列模型与注意力机制

---

上一篇： 【课程5 - 第二周编程作业】※※※※※ 【回到目录】※※※※※下一篇： 【待撰写-课程5 - 第三周编程作业】

---

1. 想一想使用如下的编码-解码模型来进行机器翻译：
   
   这个模型是“条件语言模型”,编码器部分(绿色显示)的意义是建模中输入句子x的概率

   • 正确
   • 错误

2. 在集束搜索中，如果增加集束宽度 $b$，以下哪一项是正确的？

   • 集束搜索将运行的更慢。
   • 集束搜索将使用更多的内存。
   • 集束搜索通常将找到更好地解决方案（比如：在最大化概率 $P(y|x$)上做的更好）。
   • 集束搜索将在更少的步骤后收敛。

3. 在机器翻译中，如果我们在不使用句子归一化的情况下使用集束搜索，那么算法会输出过短的译文。

   • 正确
   • 错误

4. 假设你正在构建一个能够让语音片段 $x$转为译文 $y$的基于RNN模型的语音识别系统，你的程序使用了集束搜索来试着找寻最大的 $P(y|x)$的值 $y$。在开发集样本中，给定一个输入音频，你的程序会输出译文 $\hat{y} =$ “I’m building an A Eye system in Silly con Valley.”，人工翻译为 $y^* =$ “I’m building an AI system in Silicon Valley.”

   在你的模型中,

   $P(\hat{y} \mid x) = 1.09*10^{-7}$

   $P(y^* \mid x) = 7.21*10^{-8}$

   那么，你会增加集束宽度 $B$来帮助修正这个样本吗？

   • 不会，因为 $P(y^* \mid x) \leq P(\hat{y} \mid x)$ 说明了这个锅要丢给RNN，不能让搜索算法背锅。

   • 不会，因为 $P(y^* \mid x) \leq P(\hat{y} \mid x)$ 说明了这个锅要丢给搜索算法，凭什么让RNN背锅？

   • 会的，因为 $P(y^* \mid x) \leq P(\hat{y} \mid x)$ 说明了都是RNN的错，咱不能冤枉搜索算法。

   • 会的，因为 $P(y^* \mid x) \leq P(\hat{y} \mid x)$ 说明了千错万错都是搜索算法的错，可不能惩罚RNN啊~

   博主注：皮这一下好开心~(～￣▽￣)～

5. 接着使用第4题那里的样本，假设你花了几周的时间来研究你的算法，现在你发现，对于绝大多数让算法出错的例子而言， $P(y^* \mid x) \leq P(\hat{y} \mid x)$，这表明你应该将注意力集中在改进搜索算法上，对吗？

   • 嗯嗯~
   • 不对

6. 回想一下机器翻译的模型：
   
   除此之外，还有个公式 $a^{<t,t'>} = \frac{\text{exp}(e^{<t,t'>})}{\sum^{T_x}_{t'=1}\text{exp}(e^{<t,t'>})}$

   下面关于 $\alpha^{<t,t’>}$ 的选项那个（些）是正确的？

   • 对于网络中与输出 $y^{<t>}$高度相关的 $\alpha^{<t'>}$ 而言，我们通常希望 $\alpha^{<t,t'>}$的值更大。（请注意上标）
   • 对于网络中与输出 $y^{<t>}$高度相关的 $\alpha^{<t>}$ 而言，我们通常希望 $\alpha^{<t,t'>}$的值更大。（请注意上标）
   • $\sum_{t} \alpha^{<t,t'>} = 1$ (注意是和除以t.)
   • $\sum_{t'} \alpha^{<t,t'>}=1$ (注意是和除以t′.)

7. 网络通过学习的值 $e^{<t,t'>}$来学习在哪里关注“关注点”，这个值是用一个小的神经网络的计算出来的：

   这个神经网络的输入中，我们不能将 $s^{<t>}$替换为 $s^{<t-1>}$。这是因为 $s^{<t>}$依赖于 $\alpha^{<t,t'>}$，而 $\alpha^{<t,t'>}$又依赖于 $e^{<t,t'>}$；所以在我们需要评估这个网络时，我们还没有计算出 $s^{t}$。

   • 正确
   • 错误

8. 与题1中的编码-解码模型（没有使用注意力机制）相比，我们希望有注意力机制的模型在下面的情况下有着最大的优势：

   • 输入序列的长度 $T_x$比较大。
   • 输入序列的长度 $T_x$比较小。

9.在CTC模型下，不使用"空白"字符（_）分割的相同字符串将会被折叠。那么在CTC模型下，以下字符串将会被折叠成什么样子？__c_oo_o_kk___b_ooooo__oo__kkk

• cokbok
  • cookbook
  • cook book
  • coookkboooooookkk

10. 在触发词检测中, $x^{<t>}$ 是:
    • 时间 $t$时的音频特征（就像是频谱特征一样）。
    • 第 $t$个输入字，其被表示为一个独热向量或者一个字嵌入。
    • 是否在第 $t$时刻说出了触发词。
    • 是否有人在第 $t$时刻说完了触发词。

---

Sequence models & Attention mechanism

1. Consider using this encoder-decoder model for machine translation.

This model is a “conditional language model” in the sense that the encoder portion (shown in green) is modeling the probability of the input sentence $x$.
- [x] True
- [ ] False

2. In beam search, if you increase the beam width BB, which of the following would you expect to be true? Check all that apply.
   • Beam search will run more slowly.
   • Beam search will use up more memory.
   • Beam search will generally find better solutions (i.e. do a better job maximizing P(y \mid x)P(y∣x))
   • Beam search will converge after fewer steps.

---

3. In machine translation, if we carry out beam search without using sentence normalization, the algorithm will tend to output overly short translations.
   • True
   • False

---

4. Suppose you are building a speech recognition system, which uses an RNN model to map from audio clip $x$ to a text transcript $y$. Your algorithm uses beam search to try to find the value of $y$ that maximizes $P(y \mid x)$.
   On a dev set example, given an input audio clip, your algorithm outputs the transcript $\hat{y} =$ “I’m building an A Eye system in Silly con Valley.”, whereas a human gives a much superior transcript $y^* =$ “I’m building an AI system in Silicon Valley.”.
   According to your model,
   $P(\hat{y} \mid x) = 1.09*10^{-7}$
   $P(y^∗ \mid x) = 7.21∗10^{−8}$
   Would you expect increasing the beam width B to help correct this example?

   • No, because $P(y^∗ \mid x) \leq P(\hat{y} \mid x)$ indicates the error should be attributed to the RNN rather than to the search algorithm.
   • No, because $P(y^∗ \mid x) \leq P(\hat{y} \mid x)$ indicates the error should be attributed to the search algorithm rather than to the RNN.
   • Yes, because $P(y^∗ \mid x) \leq P(\hat{y} \mid x)$ indicates the error should be attributed to the RNN rather than to the search algorithm.
   • Yes, because $P(y^∗ \mid x) \leq P(\hat{y} \mid x)$ indicates the error should be attributed to the search algorithm rather than to the RNN.

---

5. Continuing the example from Q4, suppose you work on your algorithm for a few more weeks, and now find that for the vast majority of examples on which your algorithm makes a mistake, $P(y^∗ \mid x) > P(\hat{y} \mid x)$. This suggest you should focus your attention on improving the search algorithm.
   • True
   • False

---

6. Consider the attention model for machine translation.

Further, here is the formula for $\alpha^{<t,t′>}$.

$a^{<t,t'>} = \frac{\text{exp}(e^{<t,t'>})}{\sum^{T_x}_{t'=1}\text{exp}(e^{<t,t'>})}$

Which of the following statements about $\alpha^{<t,t′>}$ are true? Check all that apply.

• We expect $\alpha^{<t,t'>}$ to be generally larger for values of $a^{<t'>}$ that are highly relevant to the value the network should output for $y^{<t>}$. (Note the indices in the superscripts.)
• We expect $\alpha^{<t,t'>}$ to be generally larger for values of $a^{<t>}$ that are highly relevant to the value the network should output for $y^{<t'>}$. (Note the indices in the superscripts.)
• $\sum_{t} \alpha^{<t,t'>}=1$ (Note the summation is over $t$.)
• $\sum_{t'} \alpha^{<t,t'>}=1$ (Note the summation is over $t'$.)

7. The network learns where to “pay attention” by learning the values e<t,t′>, which are computed using a small neural network:
   We can’t replace $s^{&lt;t-1&gt;}$ with $s^{&lt;t&gt;}$ as an input to this neural network. This is because $s^{&lt;t&gt;}$ depends on $\alpha^{&lt;t,t′&gt;}$ which in turn depends on $e^{&lt;t,t′&gt;}$; so at the time we need to evalute this network, we haven’t computed $s^{&lt;t&gt;}$ yet.

   • True
   • False

---

8. Compared to the encoder-decoder model shown in Question 1 of this quiz (which does not use an attention mechanism), we expect the attention model to have the greatest advantage when:
   • The input sequence length $T_x$ is large.
   • The input sequence length $T_x$ is small.

---

9. Under the CTC model, identical repeated characters not separated by the “blank” character (_) are collapsed. Under the CTC model, what does the following string collapse to? __c_oo_o_kk___b_ooooo__oo__kkk
   • cokbok
   • cookbook
   • cook book
   • coookkboooooookkk

---

10. In trigger word detection, $x^{&lt;t&gt;}$ is:
    • Features of the audio (such as spectrogram features) at time $t$.
    • The $t$-th input word, represented as either a one-hot vector or a word embedding.
    • Whether the trigger word is being said at time $t$.
    • Whether someone has just finished saying the trigger word at time $t$.

---