Microsoft Media Foundation官方文档翻译（16）《Extended Color Information》未完

官方英文文档链接：https://docs.microsoft.com/en-us/windows/desktop/medfound/extended-color-information

基于05/31/2018

此篇也先咕咕，感觉没完全理解，还剩一点，以后翻

In this article

1. Color Space in Media Types
2. Color Space Conversion
3. Using Extended Color Information
4. Related topics

如果你了解 RGB 颜色，那么你会知道（255, 255, 255）是白色。但是这三个 255 到底代表什么颜色？

答案可能令人惊讶：如果没有一些额外信息，这三个数字并不代表任何特定颜色。实际上任何 RGB 的颜色值都取决于色彩空间。如果我们不知道色彩空间，严格来说我们就不知道一组数字代表的颜色。

颜色空间定义了如何将一组数字表示为物理光线的颜色。当视频在一个颜色空间内编码，但显示在另一个颜色空间中时，将导致颜色扭曲，除非对视频进行色彩校正。因此，为了获得准确的颜色，了解源视频的色彩空间很重要。以前，Windows 中的视频管道不包含有关色彩空间的信息。从 Windows Vista 开始，DirectShow 和 Media Foundation 都支持在 media type 中扩展颜色信息。此信息也可用于 DirectX Video Acceleration (DXVA)。

数学上描述色彩空间的标准办法是使用 CIE XYZ 色彩空间。直接在视频中使用 CIE XYZ 是不实际的，但是 CIE XYZ 色彩空间可以在转换色彩空间时当作中间形式使用。

要准确再现颜色，需要以下信息：

• 颜色原色。颜色原色定义了 CIE XYZ 值如何表示为 RGB 颜色，the color primaries define the "meaning" of a given RGB value.
• 转换函数。这个转换函数需要把线性 RGB 值转换为非线性 R'G'B' 值。这个操作又称为伽马校正（gamma correction）。
• 转换矩阵。转换矩阵定义了 R'G'B' 如何转换为 Y'PbPr。
• 色度采样。大多数 YUV 视频中色度的分辨率低于 luma 的分辨率。色度的采样格式用 FOURCC 表示。例如，YUY2 是一种 4:2:2 格式，表示色度水平采样是每两个像素采一个样。
• Chroma siting。色度采样时，色度样本基于 luma 样本的位置决定了应该如何插值缺失的样本。
• 数值范围。表示了 Y'PbPr 值转化为 Y'CbCr 值时该如何缩放。

Color Space in Media Types

DirectShow, Media Foundation, 和 DirectX Video Acceleration (DXVA) 都有不同的方式来表示视频格式。幸运的是，很容易将色彩空间信息转换成另一个，因为相关的枚举是相同的。

• DXVA 1.0: Color-space 信息在 DXVA_ExtendedFormat 结构体中给出。

• DXVA 2.0: Color-space 信息在 DXVA2_ExtendedFormat 结构体中给出。这个结构体与 DXVA 1.0 结构相同，并且字段的含义相同。

• DirectShow: Color-space 信息在 VIDEOINFOHEADER2 结构中。信息存储在 24-bit 的 dwControlFlags 标志字段上。如果存在色彩空间信息，则 AMCONTROL_COLORINFO_PRESENT 设置在 dwControlFlags 上。当此标志位设置时，dwControlFlags 标志字段应解释为 DXVA_ExtendedFormat 结构，except that the lower 8 bits of the structure are reserved for AMCONTROL_xxx flags.

• Video capture drivers: Color-space information is given in the KS_VIDEOINFOHEADER2 structure. This structure is identical to the VIDEOINFOHEADER2 structure, and the meaning of the fields is the same.

• Media Foundation: 色彩空间信息以 attributes 存储在 media type 中：

  Color information	Attribute
  Color primaries	MF_MT_VIDEO_PRIMARIES
  Transfer function	MF_MT_TRANSFER_FUNCTION
  Transfer matrix	MF_MT_YUV_MATRIX
  Chroma subsampling	MF_MT_SUBTYPE (Given by the FOURCC, which is stored in the first DWORD of the subtype GUID.)
  Chroma siting	MF_MT_VIDEO_CHROMA_SITING
  Nominal range	MF_MT_VIDEO_NOMINAL_RANGE

Color Space Conversion

从 Y'CbCr 色彩空间转换到其他色彩空间需要以下几步。

1. Inverse quantization: 将 Y'CbCr 表示转换为 Y'PbPr 表示。

2. Upsampling: 通过插值将色度采样转换为 4:4:4。

3. YUV to RGB conversion: 使用 source transfer matrix 将 Y'PbPr 转换为非线性 R'G'B'。

4. Inverse transfer function: 反向使用转换函数将非线性 R'G'B' 转换为线性 RGB。

5. RGB color space conversion: Use the color primaries to convert from the source RGB space to the target RGB space.

6. Transfer function: Convert linear RGB to non-linear R'G'B, using the target transfer function.

7. RGB to YUV conversion: Convert R'G'B' to Y'PbPr, using the target transfer matrix.

8. Downsampling: Convert 4:4:4 to 4:2:2, 4:2:0, or 4:1:1 by filtering the chroma values.

9. Quantization: Convert Y'PbPr to Y'CbCr, using the target nominal range.

Steps 1–4 occur in the source color space, and steps 6–9 occur in the target color space. In the actual implementation, intermediate steps can be approximated and adjacent steps can be combined. There is generally a trade-off between accuracy and computational cost.

For example, to convert from RT.601 to RT.709 requires the following stages:

• Inverse quantization: Y'CbCr(601) to Y'PbPr(601)

• Upsampling: Y'PbPr(601)

• YUV to RGB: Y'PbPr(601) to R'G'B'(601)

• Inverse transfer function: R'G'B'(601) to RGB(601)

• RGB color space conversion: RGB(601) to RGB(709)

• Transfer function: RGB(709) to R'G'B'(709)

• RGB to YUV: R'G'B'(709) to Y'PbPr(709)

• Downsampling: Y'PbPr(709)

• Quantization: Y'PbPr(709) to Y'CbCr(709)

Using Extended Color Information

To preserve color fidelity throughout the pipeline, color-space information must be introduced at the source or the decoder and conveyed all the way through the pipeline to the sink.

Video Capture Devices

Most analog capture devices use a well-defined color space when capturing video. Capture drivers should offer a format with a KS_VIDEOINFOHEADER2 format block that contains the color information. For backward compatibility, the driver should accept formats that do not contain the color information. This will enable the driver to work with components that do not accept the extended color information.

File-based Sources

When parsing a video file, the media source (or parser filter, in DirectShow) might be able to provide some of the color information. For example, the DVD Navigator can determine the color space based on the DVD content. Other color information might be available to the decoder. For example, an MPEG-2 elementary video stream gives the color information in the sequence_display_extension field. If the color information is not explicitly described in the source, it might be defined implicitly by the type of content. For example, the NTSC and PAL varieties of DV video each use different color spaces. Finally, the decoder can use whatever color information it gets from the source's media type.

Other Components

Other components might need to use the color-space information in a media type:

• Software color-space converters should use color-space information when selecting a conversion algorithm.
• Video mixers, such as the enhanced video renderer (EVR) mixer, should use the color information when mixing video streams from different types of content.
• The DXVA video processing APIs and DDIs enable the caller to specify color-space information information. The GPU should use this information when it performs hardward video mixing.

Related topics