A Preliminary Study on HTML Video Video Subtitles

What are external subtitles?

When we watch TV and movies, we can see that under the playback area, the subtitles corresponding to the characters in the video will be displayed. Subtitles provide translations for content that viewers cannot understand. For example, Chinese subtitles can be displayed in English movies.

Many early videos are adopted 「内嵌字幕」: video files and subtitle files are integrated together. Instead 「外挂字幕」, the video file is separated from the subtitle file, and the subtitle file is poured into the subtitle file for display during playback.

The advantage of external subtitles is that the same video can support importing and presenting languages of multiple countries, and supports switching subtitles at any time.

HTML Video tag uses external subtitles

On the front end, we use HTML <Video />the tag to implement video playback. To use external subtitles, just use <track>the tag to specify subtitles for the video.

The HTML element will be used as a sub-element of the <track>media elements <audio>and , which allow specifying a timed text subtitle file (or time-based data), and automatically display subtitles at the appropriate time.<video>

It has a requirement for the subtitle format file: needs to be WebVTT 格式（.vtt 后缀文件）.

Here is a simple example of a video playing with subtitles:

<video controls>
  <source src="./video.mp4">
  <track src="./video.vtt" kind="subtitles" srclang="en" label="English" default="true">
</video>

Note: The above sample code needs to run on a server, or use VSCode to open by starting a local service. If you open it as file://Users... you will see an error like: 'file:' URLs are treated as unique security origins..

Among them, <source> srcit is used to specify the video source, and <track> srcis used to specify the external subtitle file. The content of the .vtt subtitle file is as follows:

WEBVTT

00:00:00.000 --> 00:00:05.000
This is the first subtitle.

00:00:05.000 --> 00:00:10.000
This is the second subtitle.

It will display the first subtitle within 0 - 5s of video playback, and the second subtitle within 5 - 10s. The rendering is as follows:

<track>In addition to the src attribute, the tag rendering subtitle file also has several important attributes:

kindAttribute, which defines the format in which the text is displayed. The default is subtitlesto use subtitles. Other keywords allowed are: captions, descriptions, chapters, metadata.
default 属性，定义了默认要启用哪一个字幕文件；一般一个视频会提供多种语言的字幕文件。
label 属性，字幕文件呈现给用户可读的标题；
srclang 属性，字幕文件所属语种。

了解 Web 视频文本轨道（WebVTT）

在上面，我们为 <track> src 指向的字幕资源格式为 .vtt 字幕文件。

.vtt 格式文件属于 Web Video Text Tracks Format（WebVTT），是一种基于文本 UTF-8 编码的格式，为 Web 媒体元素提供字幕数据文件。WebVTT 的 MIME 格式是 text/vtt。

1、如何编写 WebVTT 格式内容

一个 WebVTT 文件（.vtt）以 WEBVTT 作为开头，包含一个或者多个的 时间提示性内容（cue），如下所示：

WEBVTT

00:01.000 --> 00:04.000
Never drink liquid <b>nitrogen</b>.

00:05.000 --> 00:09.000
- It will perforate your stomach.
- You could die.

每个 cue 的作用如下：

第一行决定字幕以时间开始展示 --> 到什么时间结束展示；
可以有一个或多个行，每个行都是展示的字幕的一部分。

2、如何为 WebVTT 中的字幕添加样式

CSS 提供了查找与 ::cue 伪元素 匹配的元素来为 WebVTT 添加样式。如下：

video::cue {
  background-image: linear-gradient(to bottom, dimgray, lightgray);
  color: papayawhip;
}

video::cue(b) {
  color: peachpuff;
}

在这里，所有字幕风格都使用一个灰色的线性渐变作为它们的背景，字体颜色为 "papayawhip"。此外，使用 <b> 元素加粗的字幕使用 "peachpuff" 颜色。

允许为 WebVTT 字幕设置的 CSS 样式属性参考：

color
opacity
visibility
text-decoration及相关属性
text-shadow
background及相关属性
outline及相关属性
font及相关属性，包括line-height
white-space
text-combine-upright
ruby-position

另外，在 WebVTT 文件中，每个 cue 同时支持设置在视频中的具体位置，如将第一个字幕居中展示在顶部，第二个字幕居中展示在底部，示例如下：

WEBVTT

00:01.000 --> 00:04.000 line:0% position:50% align:center
Never drink liquid <b>nitrogen</b>.

00:05.000 --> 00:09.000 line:100% position:50% align:center
- It will perforate your stomach.
- You could die.

此外，cue 内容同样支持一些文本标签一起使用，具体介绍可查看 WebVTT

自定义 Video 播放器字幕

当在 <video> 标签中加入 <track> 字幕标签后，video 原生 controls 功能栏中会展示字幕入口。

然而，视频播放器通常会根据业务需求及 UI 样式采用自定义方式实现，下面我们来看看自定义播放器中如何集成字幕功能并实现多字幕切换。

字幕切换列表的配置我们这样设计：由外部传入 <track> 标签所需的信息，如下，

config = {
  ...,
  nativeTextTrack: [
    {
      src: "./vtt/textTrack-1.vtt",
      kind: "subtitles",
      label: "中文字幕",
      srclang: "zh",
      default: true,
    },
    {
      src: "./vtt/textTrack-2.vtt",
      kind: "subtitles",
      label: "英文字幕",
      srclang: "en",
      default: false,
    },
  ],
}

// textTrack 类型接口参考
export interface TextTrack {
  src: string;
  kind: string;
  label: string;
  srclang: string;
  default: boolean;
}

这里我们定义了视频的两种不同语种字幕，下面基于这个字幕配置信息，为 <video> 标签创建 <track> 字幕元素，这个逻辑一般在初始化播放器时来实现。

const { nativeTextTrack } = config; // 外部传入的配置

// 字幕
let textTrackDom = '';
if (Array.isArray(nativeTextTrack) && nativeTextTrack.length > 0) {
  // 应用默认字幕
  if (!nativeTextTrack.some(track => track.default)) {
    nativeTextTrack[0].default = true;
  }
  // 创建 <track> 字幕标签
  nativeTextTrack.forEach(track => {
    if (track.src && track.label && track.srclang) {
      textTrackDom += `<track src="${track.src}" kind="${track.kind || 'subtitles'}" label="${track.label}" srclang="${track.srclang}" ${track.default ? 'default' : ''}>`;
    }
  });
}

// 播放器 video 元素实例，添加 <track> 为子元素
video.appendChild(textTrackDom);

添加后看到的 video 元素内部结构如下：

接下来，我们需要在 Controls 区域增加字幕切换入口，并且渲染出切换字幕的选项列表，列表中默认包含一个可供 Off 关闭字幕 的选项。

const { state, config, controls, root, video } = player;

const textTrackArea = createDOM(
  'div',
  `
    <div class="dsplayer-texttrack-name">字幕</div> // 字幕在 Controls 中展示的信息
    <ul class="dsplayer-subtitles-menu"> // 字幕选项列表
      <li class="dsplayer-subtitles-menu__item" lang="off">Off</li>
    </ul>
  `,
  {},
  'dsplayer-control-texttrack',
);
controls.appendChild(textTrackArea); // 添加到 Controls 区域

const textTrackMenu = textTrackArea.querySelector('.dsplayer-subtitles-menu') as HTMLElement;
// 根据 nativeTextTrack 创建选项列表
config.nativeTextTrack.forEach(track => {
  textTrackMenu.appendChild(createDOM('li', track.label, { lang: track.srclang }, 'dsplayer-subtitles-menu__item'));
});

要实现切换字幕（切换为另一种语种字幕），可通过设定 video.track 元素的 mode 属性来控制。mode = showing 代表当前要使用的字幕，mode = hidden 则是将当前字幕隐藏。参考示例代码如下：

// 事件委托
textTrackMenu.addEventListener('click', event => {
  const ele = event.target as HTMLElement;
  if (ele.nodeName === 'LI') {
    const lang = ele.getAttribute('lang');
    for (let i = 0; i < player.video.textTracks.length; i ++) {
      const track = player.video.textTracks[i];
      if (track.language === lang) {
        track.mode = 'showing'; // 应用字幕
      } else {
        track.mode = 'hidden'; // 隐藏字幕
      }
    }
  }
});

文末

有一些情况下提供的字幕文件是 .srt 格式，无法直接被使用，通常都需要做一次 vtt 字幕格式转换。可以通过在线工具来完成。

参考

1. mdn web docs
2. mdn web docs WebVTT
3. 常见的字幕文件格式
 4. 玩转HTML5 Video视频WebVTT字幕使用样式与制作
 5. 向 HTML 视频中添加字幕
 6. xgplayer 外挂字幕