官网：https://gstreamer.freedesktop.org/
文档教程：https://gstreamer.freedesktop.org/documentation/plugin-development/index.html
目录
GstMiniObject, Buffers and Events
GStreamer是创建流媒体应用程序的一个非常强大和通用的框架。GStreamer框架的许多优点来自于它的模块化:GStreamer可以无缝地集成新的插件模块。但是，由于模块化和功能强大的代价往往是更高的复杂性(考虑一下，例如CORBA)，编写新的插件并不总是容易的。
本指南旨在帮助您理解GStreamer框架，以便开发新的插件来扩展现有功能。教程通过一个示例插件(C语言实现的一个音频过滤器插件)来阐述接下来的大部分问题.然而,指南在之后的部分介绍编写其他类型的插件,指南的结束位置描述了如果通过Python来绑定GStreamer。
GStreamer is an extremely powerful and versatile framework for creating streaming media applications. Many of the virtues of the GStreamer framework come from its modularity: GStreamer can seamlessly incorporate new plugin modules. But because modularity and power often come at a cost of greater complexity (consider, for example, CORBA), writing new plugins is not always easy.
This guide is intended to help you understand the GStreamer framework so you can develop new plugins to extend the existing functionality. The guide addresses most issues by following the development of an example plugin - an audio filter plugin - written in C. However, the later parts of the guide also present some issues involved in writing other types of plugins, and the end of the guide describes some of the Python bindings for GStreamer.
Preface
What is GStreamer?
GStreamer是一个用于创建流媒体应用程序的框架。基础设计来自俄勒冈研究生院的视频管道，以及DirectShow的一些想法。
GStreamer的开发框架可以编写任何类型的流式多媒体应用程序。GStreamer框架旨在简化编写处理音频或视频或两者的应用程序。它不仅限于音频和视频，还可以处理任何类型的数据流。管道设计的开销小于所应用的过滤器引起的开销。这使得GStreamer成为设计高端音频应用程序的良好框架，这些应用程序对延迟或性能提出了很高的要求。
GStreamer最明显的用途之一是使用它来构建媒体播放器。GStreamer已经包含用于构建媒体播放器的组件，可以支持各种格式， 包括MP3，Ogg / Vorbis，MPEG-1/2，AVI，Quicktime，mod等。然而，GStreamer不仅仅是另一个媒体播放器。它的主要优点是可插拔组件可以混合并匹配到任意管道中，因此可以编写完整的视频或音频编辑应用程序。
该框架基于提供各种编解码器和其他功能的插件。插件可以链接并布置在管道中。此管道定义数据流。
GStreamer核心功能是为插件，数据流，同步和媒体类型处理/协商提供了一个框架。它还提供了使用各种插件编写应用程序的API。
GStreamer is a framework for creating streaming media applications. The fundamental design comes from the video pipeline at Oregon Graduate Institute, as well as some ideas from DirectShow.
GStreamer's development framework makes it possible to write any type of streaming multimedia application. The GStreamer framework is designed to make it easy to write applications that handle audio or video or both. It isn't restricted to audio and video, and can process any kind of data flow. The pipeline design is made to have little overhead above what the applied filters induce. This makes GStreamer a good framework for designing even high-end audio applications which put high demands on latency or performance.
One of the most obvious uses of GStreamer is using it to build a media player. GStreamer already includes components for building a media player that can support a very wide variety of formats, including MP3, Ogg/Vorbis, MPEG-1/2, AVI, Quicktime, mod, and more. GStreamer, however, is much more than just another media player. Its main advantages are that the pluggable components can be mixed and matched into arbitrary pipelines so that it's possible to write a full-fledged video or audio editing application.
The framework is based on plugins that will provide the various codec and other functionality. The plugins can be linked and arranged in a pipeline. This pipeline defines the flow of the data.
The GStreamer core function is to provide a framework for plugins, data flow, synchronization and media type handling/negotiation. It also provides an API to write applications using the various plugins.
Who Should Read This Guide?
本指南介绍了如何为GStreamer编写新模块。该指南与几组人群相关：
-
任何想要对GStreamer添加新方法的人。例如，该组中的某个人可能想要创建新的数据格式转换器，新的可视化工具或新的解码器或编码器。
-
任何想要添加对新输入和输出设备的支持的人。例如，该组中的人可能希望添加写入新视频输出系统或从数码相机或特殊麦克风读取数据的功能。
-
任何想要以任何方式扩展GStreamer的人。在理解插件系统对其余代码的约束之前，您需要了解插件系统的工作原理。此外，阅读本文后，您可能会对使用插件可以完成的工作感到惊讶。
如果您只想使用GStreamer的现有功能，或者您只想使用GStreamer的应用程序，则本指南与您无关。如果您只对使用现有插件编写新应用程序感兴趣 - 而且已经有相当多的插件 - 您可能需要查看GStreamer应用程序开发手册。如果您只是想获得GStreamer应用程序的帮助，那么您应该查看该特定应用程序的用户手册。
This guide explains how to write new modules for GStreamer. The guide is relevant to several groups of people:
-
Anyone who wants to add support for new ways of processing data in GStreamer. For example, a person in this group might want to create a new data format converter, a new visualization tool, or a new decoder or encoder.
-
Anyone who wants to add support for new input and output devices. For example, people in this group might want to add the ability to write to a new video output system or read data from a digital camera or special microphone.
-
Anyone who wants to extend GStreamer in any way. You need to have an understanding of how the plugin system works before you can understand the constraints that the plugin system places on the rest of the code. Also, you might be surprised after reading this at how much can be done with plugins.
This guide is not relevant to you if you only want to use the existing functionality of GStreamer, or if you just want to use an application that uses GStreamer. If you are only interested in using existing plugins to write a new application - and there are quite a lot of plugins already - you might want to check the GStreamer Application Development Manual. If you are just trying to get help with a GStreamer application, then you should check with the user manual for that particular application.
Preliminary Reading
本指南假设您对GStreamer的基本工作方式有所了解。有关GStreamer中编程概念的简要介绍，您可能应该首先阅读GStreamer应用程序开发手册。另请查看GStreamer网站上提供的其他文档。
为了理解本手册，您需要对C语言有基本的了解。由于GStreamer遵循GObject编程模型，本指南还假设您了解GObject 编程的基础知识。您还可以参考Eric Harlow写的书的书“ 使用GTK +和GDK开发Linux应用程序”。
This guide assumes that you are somewhat familiar with the basic workings of GStreamer. For a gentle introduction to programming concepts in GStreamer, you may wish to read the GStreamer Application Development Manual first. Also check out the other documentation available on the GStreamer web site.
In order to understand this manual, you will need to have a basic understanding of the C language. Since GStreamer adheres to the GObject programming model, this guide also assumes that you understand the basics of GObject programming. You may also want to have a look at Eric Harlow's book Developing Linux Applications with GTK+ and GDK.
Structure of This Guide
为了帮助您浏览本指南，它分为几个大部分。每个部分都涉及有关GStreamer插件开发的特定主题。本指南的各部分按以下顺序排列：
To help you navigate through this guide, it is divided into several large parts. Each part addresses a particular broad topic concerning GStreamer plugin development. The parts of this guide are laid out in the following order:
本部分介绍了构建插件通常需要执行的所有基本步骤，例如使用GStreamer注册元素以及设置基础知识，以便它可以从邻居元素接收数据并将数据发送到相邻元素。讨论首先给出了生成基本结构和在Constructing the Boilerplate中注册元素 的示例。然后，您将学习如何编写代码以获得一个基本的过滤器插件，这些插件工作在特定pads，链函数 和那些状态。
之后，我们将展示一些关于如何为应用程序配置元素以及如何在添加属性和信号中进行应用程序元素交互的GObject概念。接下来，您将学习如何构建一个快速测试应用程序来测试您在构建测试应用程序中学到的所有知识。我们将在这里介绍基础知识。对于成熟的应用程序开发，您应该查看“应用程序开发手册”。
-
Building a Plugin - Introduction to the structure of a plugin, using an example audio filter for illustration.
This part covers all the basic steps you generally need to perform to build a plugin, such as registering the element with GStreamer and setting up the basics so it can receive data from and send data to neighbour elements. The discussion begins by giving examples of generating the basic structures and registering an element in Constructing the Boilerplate. Then, you will learn how to write the code to get a basic filter plugin working in Specifying the pads, The chain function and What are states?.
After that, we will show some of the GObject concepts on how to make an element configurable for applications and how to do application-element interaction in Adding Properties and Signals. Next, you will learn to build a quick test application to test all that you've just learned in Building a Test Application. We will just touch upon basics here. For full-blown application development, you should look at the Application Development Manual.
-
高级过滤器概念 - 有关GStreamer插件开发的高级功能的信息。
在了解了基本步骤之后，您应该能够创建具有一些不错功能的功能音频或视频过滤器插件。但是，GStreamer为插件编写者提供了更多功能。本指南的这一部分包括有关更高级主题的章节，例如调度，GStreamer中的媒体类型定义，时钟，接口和标记。由于这些功能是针对特定目的的，因此您可以按任何顺序阅读它们，其中大多数不需要其他部分的知识。
第一章，名为不同的调度模式，解释元素调度的一些基础知识。它不是很深入，但主要是介绍其他事情为何如此起作用。如果您对GStreamer内部感兴趣，请阅读本章。接下来，我们将应用这些知识并讨论另一种类型的数据传输不同的调度模式（而不是您在链函数中学到的）。基于循环的元素将使您更好地控制输入速率。这在编写复用器或解复用器时非常有用。
接下来，我们将在媒体类型和属性中讨论GStreamer中的媒体识别。您将学习如何定义新媒体类型并了解GStreamer中定义的标准媒体类型列表。
在下一章中，您将学习request pads和sometimes pads的概念，这些垫是动态创建的垫，因为应用程序要求它（请求）或因为媒体流需要它（有时）。
下一章Clocking将解释GStreamer中时钟的概念。当您想知道元素应该如何实现音频/视频同步时，您需要此信息。
接下来的几章将讨论进行应用程序元素交互的高级方法。以前，我们在GObject中了解了添加属性和 信号的方法。我们将在支持动态参数中讨论动态参数，这是一种预先定义元素行为的方法。接下来，您将了解Interfaces中的接口。基于GObject的GInterface，接口是应用程序元素交互的特定于目标的方式。最后，您将了解在GStreamer中元数据如何在被处理在标记（元数据和Streaminfo）中。
最后一章，活动：寻求，导航等，将讨论GStreamer中事件的概念。事件是进行应用程序元素交互的另一种方式。例如，他们负责寻求。它们也是元素彼此交互的另一种方式，例如让对方了解媒体流不连续性，在管道内转发标签等等。
-
Advanced Filter Concepts - Information on advanced features of GStreamer plugin development.
After learning about the basic steps, you should be able to create a functional audio or video filter plugin with some nice features. However, GStreamer offers more for plugin writers. This part of the guide includes chapters on more advanced topics, such as scheduling, media type definitions in GStreamer, clocks, interfaces and tagging. Since these features are purpose-specific, you can read them in any order, most of them don't require knowledge from other sections.
The first chapter, named Different scheduling modes, will explain some of the basics of element scheduling. It is not very in-depth, but is mostly some sort of an introduction on why other things work as they do. Read this chapter if you're interested in GStreamer internals. Next, we will apply this knowledge and discuss another type of data transmission than what you learned in The chain function: Different scheduling modes. Loop-based elements will give you more control over input rate. This is useful when writing, for example, muxers or demuxers.
Next, we will discuss media identification in GStreamer in Media Types and Properties. You will learn how to define new media types and get to know a list of standard media types defined in GStreamer.
In the next chapter, you will learn the concept of request- and sometimes-pads, which are pads that are created dynamically, either because the application asked for it (request) or because the media stream requires it (sometimes). This will be in Request and Sometimes pads.
The next chapter, Clocking, will explain the concept of clocks in GStreamer. You need this information when you want to know how elements should achieve audio/video synchronization.
The next few chapters will discuss advanced ways of doing application-element interaction. Previously, we learned on the GObject-ways of doing this in Adding Properties and Signals. We will discuss dynamic parameters, which are a way of defining element behaviour over time in advance, in Supporting Dynamic Parameters. Next, you will learn about interfaces in Interfaces. Interfaces are very target- specific ways of application-element interaction, based on GObject's GInterface. Lastly, you will learn about how metadata is handled in GStreamer in Tagging (Metadata and Streaminfo).
The last chapter, Events: Seeking, Navigation and More, will discuss the concept of events in GStreamer. Events are another way of doing application-element interaction. They take care of seeking, for example. They are also yet another a way in which elements interact with each other, such as letting each other know about media stream discontinuities, forwarding tags inside a pipeline and so on.
-
创建特殊元素类型 - 编写其他插件类型的说明。
由于本指南的前两部分使用音频过滤器作为示例，因此引入的概念适用于过滤器插件。但是许多概念同样适用于其他插件类型，包括源，接收器和自动插件。本指南的这一部分介绍了处理这些更专业的插件类型时出现的问题。本章首先重点介绍可以使用基类（预制基类）编写的元素，然后在写入解复用器或解析器时编写特殊类型的元素，编写N对1元素或复用器 和编写管理器。
-
Creating special element types - Explanation of writing other plugin types.
Because the first two parts of the guide use an audio filter as an example, the concepts introduced apply to filter plugins. But many of the concepts apply equally to other plugin types, including sources, sinks, and autopluggers. This part of the guide presents the issues that arise when working on these more specialized plugin types. The chapter starts with a special focus on elements that can be written using a base-class (Pre-made base classes), and later also goes into writing special types of elements in Writing a Demuxer or Parser, Writing a N-to-1 Element or Muxer and Writing a Manager.
-
附录 - 插件开发人员的更多信息。这些附录包含一些顽固地拒绝在指南的其他部分中干净利落的信息。本节的大部分内容尚未完成。
-
Appendices - Further information for plugin developers.
The appendices contain some information that stubbornly refuses to fit cleanly in other sections of the guide. Most of this section is not yet finished.
本指南的前言部分的其余部分简要概述了GStreamer插件开发中涉及的基本概念。涵盖的主题包括元素和插件， Pads，GstMiniObject，缓冲区和事件 以及媒体类型和属性。如果您已经熟悉此信息，可以使用此简短概述来刷新内存，或者跳转到构建插件。
如您所见，有很多东西需要学习，所以让我们开始吧！
-
通过从GstBin扩展来创建复合元素和复杂元素。这将允许您创建插入其他插件的插件。
-
将新媒体类型与typedetect函数一起添加到注册表中。这将允许您的插件在全新的媒体类型上运行。
The remainder of this introductory part of the guide presents a short overview of the basic concepts involved in GStreamer plugin development. Topics covered include Elements and Plugins, Pads, GstMiniObject, Buffers and Events and Media types and Properties. If you are already familiar with this information, you can use this short overview to refresh your memory, or you can skip to Building a Plugin.
As you can see, there a lot to learn, so let's get started!
-
Creating compound and complex elements by extending from a GstBin. This will allow you to create plugins that have other plugins embedded in them.
-
Adding new media types to the registry along with typedetect functions. This will allow your plugin to operate on a completely new media type.
Foundations
本指南的这一章介绍了GStreamer的基本概念。理解这些概念将有助于您解决扩展GStreamer所涉及的问题。许多这些概念在GStreamer应用程序开发手册中有更详细的解释; 这里介绍的基本概念主要用于刷新你的记忆。
This chapter of the guide introduces the basic concepts of GStreamer. Understanding these concepts will help you grok the issues involved in extending GStreamer. Many of these concepts are explained in greater detail in the GStreamer Application Development Manual; the basic concepts presented here serve mainly to refresh your memory.
Elements and Plugins
元素是GStreamer的核心。在插件开发的上下文中，元素是从GstElement类派生的对象。元素在与其他元素链接时提供某种功能：例如，源元素向流提供数据，过滤元素作用于流中的数据。没有元素，GStreamer只是一堆没有任何链接的概念管道配件。GStreamer附带了大量元素，但也可以编写额外的元素。
但是，仅仅编写新元素是不行的，您还需要将元素封装在插件中以使GStreamer能够使用它。插件本质上是一个可加载的代码块，通常称为共享对象文件或动态链接库。单个插件可能包含多个元素的实现，或者只包含一个元素。为简单起见，本指南主要关注包含一个元素的插件。
滤波器是元件的一个重要类型处理数据的流。数据的生产者和消费者分别称为源和宿 元素。Bin元素包含其他元素。一种类型的bin负责同步它们包含的元素，以便数据流畅地流动。另一种类型的bin，称为 autoplugger元素，自动将其他元素添加到bin并将它们链接在一起，以便它们充当两个任意流类型之间的过滤器。
即使只使用标准软件包，插件机制也可在GStreamer中随处使用。一些非常基本的函数驻留在核心库中，所有其他函数都在插件中实现。插件注册表用于在二进制注册表文件中存储插件的详细信息。这样，使用GStreamer的程序不必加载所有插件来确定所需的插件。只有在元素请求的时候才会加载插件。
关于GStreamer更详细信息参考参考GstElement和 GstPlugin。
Elements are at the core of GStreamer. In the context of plugin development, an element is an object derived from the GstElement class. Elements provide some sort of functionality when linked with other elements: For example, a source element provides data to a stream, and a filter element acts on the data in a stream. Without elements, GStreamer is just a bunch of conceptual pipe fittings with nothing to link. A large number of elements ship with GStreamer, but extra elements can also be written.
Just writing a new element is not entirely enough, however: You will need to encapsulate your element in a pluginto enable GStreamer to use it. A plugin is essentially a loadable block of code, usually called a shared object file or a dynamically linked library. A single plugin may contain the implementation of several elements, or just a single one. For simplicity, this guide concentrates primarily on plugins containing one element.
A filter is an important type of element that processes a stream of data. Producers and consumers of data are called source and sink elements, respectively. Bin elements contain other elements. One type of bin is responsible for synchronization of the elements that they contain so that data flows smoothly. Another type of bin, calledautoplugger elements, automatically add other elements to the bin and links them together so that they act as a filter between two arbitrary stream types.
The plugin mechanism is used everywhere in GStreamer, even if only the standard packages are being used. A few very basic functions reside in the core library, and all others are implemented in plugins. A plugin registry is used to store the details of the plugins in an binary registry file. This way, a program using GStreamer does not have to load all plugins to determine which are needed. Plugins are only loaded when their provided elements are requested.
See the GStreamer Library Reference for the current implementation details of GstElement and GstPlugin.
Pads
Pads用于协商GStreamer中元素之间的链接和数据流。可以将垫视为元素上的“位置”或“端口”，其中可以与其他元素建立链接，并且数据可以通过该元素流入或流出这些元素。Pads具有特定的数据处理功能：pad可以限制流经它的数据类型。当两个pad的允许数据类型兼容时，才允许在两个pad之间建立链接。
这里的比喻可能会有所帮助。pad类似于物理设备上的插头或插孔。例如，考虑由放大器，DVD播放器和（静音）视频投影仪组成的家庭影院系统。允许将DVD播放器连接到放大器，因为两个设备都有音频插孔，并且允许将投影仪连接到DVD播放器，因为两个设备都具有兼容的视频插孔。由于投影仪和放大器具有不同类型的插孔，因此可能无法建立投影仪和放大器之间的链接。GStreamer中的垫子与家庭影院系统中的插孔具有相同的用途。
在大多数情况下，GStreamer中的所有数据都通过元素之间的链接流动。数据通过一个或多个source pad流出，通过一个或多个sink pad接收数据。source element仅仅含有src pad，sink element 仅仅含有sink pad。
请参阅GStreamer库参考GstPad以获得关于这些的详细信息，
Pads are used to negotiate links and data flow between elements in GStreamer. A pad can be viewed as a “place” or “port” on an element where links may be made with other elements, and through which data can flow to or from those elements. Pads have specific data handling capabilities: A pad can restrict the type of data that flows through it. Links are only allowed between two pads when the allowed data types of the two pads are compatible.
An analogy may be helpful here. A pad is similar to a plug or jack on a physical device. Consider, for example, a home theater system consisting of an amplifier, a DVD player, and a (silent) video projector. Linking the DVD player to the amplifier is allowed because both devices have audio jacks, and linking the projector to the DVD player is allowed because both devices have compatible video jacks. Links between the projector and the amplifier may not be made because the projector and amplifier have different types of jacks. Pads in GStreamer serve the same purpose as the jacks in the home theater system.
For the most part, all data in GStreamer flows one way through a link between elements. Data flows out of one element through one or more source pads, and elements accept incoming data through one or more sink pads. Source and sink elements have only source and sink pads, respectively.
See the GStreamer Library Reference for the current implementation details of a GstPad.
GstMiniObject, Buffers and Events
GStreamer中的所有数据流都被切割成块，这些块从一个元素上的src pad传递到另一个元素上的sink pad。GstMiniObject是用于保存这些数据块的结构。
GstMiniObject包含以下重要类型：
-
一个确切的类型，指示此GstMiniObject是什么类型的数据（事件，缓冲区......）。
-
引用计数，指示当前持有对miniobject的引用的元素数。当引用计数降至零时，此对象会被处理，并且在某种意义上将释放其存储器（有关详细信息，请参见下文）。
对于数据传输，定义了两种类型的GstMiniObject：事件（控制）和缓冲区（内容）。
缓冲区可能包含两个链接的打击垫知道如何处理的任何类型的数据。通常，缓冲区包含从一个元素流向另一个元素的某种音频或视频数据。
缓冲区还包含描述缓冲区内容的元数据。一些重要的元数据类型是：
-
指向一个或多个GstMemory对象的指针。GstMemory对象是封装内存区域的refcounted对象。
-
一个时间戳，指示缓冲区中内容的首选显示时间戳。
事件包含有关在两个链接的pads之间流动的流的状态的信息。仅当元素明确支持它们时才会发送事件，否则核心将（尝试）自动处理事件。事件用于指示例如媒体类型，媒体流的结尾或者应该刷新高速缓存。
事件可能包含以下几项：
-
指示所包含事件类型的子类型。
-
事件的其他内容取决于特定的事件类型。
活动将在活动中广泛讨论：寻求，导航等。在此之前，将使用的唯一事件是EOS事件，该事件用于指示流末尾（通常是文件结束）。
更详细的内容参考GStreamer库索引 GstMiniObject， GstBuffer和 GstEvent。
All streams of data in GStreamer are chopped up into chunks that are passed from a source pad on one element to a sink pad on another element. GstMiniObject is the structure used to hold these chunks of data.
GstMiniObject contains the following important types:
-
An exact type indicating what type of data (event, buffer, ...) this GstMiniObject is.
-
A reference count indicating the number of elements currently holding a reference to the miniobject. When the reference count falls to zero, the miniobject will be disposed, and its memory will be freed in some sense (see below for more details).
For data transport, there are two types of GstMiniObject defined: events (control) and buffers (content).
Buffers may contain any sort of data that the two linked pads know how to handle. Normally, a buffer contains a chunk of some sort of audio or video data that flows from one element to another.
Buffers also contain metadata describing the buffer's contents. Some of the important types of metadata are:
-
Pointers to one or more GstMemory objects. GstMemory objects are refcounted objects that encapsulate a region of memory.
-
A timestamp indicating the preferred display timestamp of the content in the buffer.
Events contain information on the state of the stream flowing between the two linked pads. Events will only be sent if the element explicitly supports them, else the core will (try to) handle the events automatically. Events are used to indicate, for example, a media type, the end of a media stream or that the cache should be flushed.
Events may contain several of the following items:
-
A subtype indicating the type of the contained event.
-
The other contents of the event depend on the specific event type.
Events will be discussed extensively in Events: Seeking, Navigation and More. Until then, the only event that will be used is the EOS event, which is used to indicate the end-of-stream (usually end-of-file).
See the GStreamer Library Reference for the current implementation details of a GstMiniObject, GstBuffer andGstEvent.
Buffer Allocation
缓冲区能够存储几种不同类型的内存块。最通用的缓冲区包含malloc（）分配的内存。这些缓冲区虽然方便，但并不总是非常快，因为数据通常需要专门复制到缓冲区中。
许多专用元素创建指向特殊内存的缓冲区。例如，file src元素通常将文件映射到应用程序的地址空间（使用mmap（）），并创建指向该地址范围的缓冲区。file src创建的这些缓冲区与通用缓冲区完全相同，只是它们是只读的。缓冲区释放代码自动确定释放底层内存的正确方法。接收这些缓冲区的下游元素不需要做任何特殊的处理或不引用它。
元素可能获得专用缓冲区的另一种方法是通过GstBufferPool或GstAllocator从下游对等方请求它们。元素可以从下游对等元素中请求GstBufferPool或GstAllocator。如果下游能够提供这些对象，则上游可以使用它们来分配缓冲区。查看更多的内存分配。
许多接收器元件具有将数据复制到硬件或直接访问硬件的加速方法。这些元素通常能够为其上游对等体创建GstBufferPool或GstAllocator。一个这样的例子是ximage sink。它创建包含XImages的缓冲区。因此，当上游对等体将数据复制到缓冲区时，它会直接复制到XImage中，使ximage sink能够将图像直接绘制到屏幕上，而不必先将数据复制到XImage中。
过滤元素通常有机会在就地处理缓冲区，或者在从源缓冲区复制到目标缓冲区时工作。实现这两种算法是最佳的，因为GStreamer框架可以根据需要选择最快的算法。当然，这只适用于严格的过滤器 - 在源和接收器垫上具有完全相同格式的元素。
Buffers are able to store chunks of memory of several different types. The most generic type of buffer contains memory allocated by malloc(). Such buffers, although convenient, are not always very fast, since data often needs to be specifically copied into the buffer.
Many specialized elements create buffers that point to special memory. For example, the filesrc element usually maps a file into the address space of the application (using mmap()), and creates buffers that point into that address range. These buffers created by filesrc act exactly like generic buffers, except that they are read-only. The buffer freeing code automatically determines the correct method of freeing the underlying memory. Downstream elements that receive these kinds of buffers do not need to do anything special to handle or unreference it.
Another way an element might get specialized buffers is to request them from a downstream peer through a GstBufferPool or GstAllocator. Elements can ask a GstBufferPool or GstAllocator from the downstream peer element. If downstream is able to provide these objects, upstream can use them to allocate buffers. See more in Memory allocation.
Many sink elements have accelerated methods for copying data to hardware, or have direct access to hardware. It is common for these elements to be able to create a GstBufferPool or GstAllocator for their upstream peers. One such example is ximagesink. It creates buffers that contain XImages. Thus, when an upstream peer copies data into the buffer, it is copying directly into the XImage, enabling ximagesink to draw the image directly to the screen instead of having to copy data into an XImage first.
Filter elements often have the opportunity to either work on a buffer in-place, or work while copying from a source buffer to a destination buffer. It is optimal to implement both algorithms, since the GStreamer framework can choose the fastest algorithm as appropriate. Naturally, this only makes sense for strict filters -- elements that have exactly the same format on source and sink pads.
Media types and Properties
GStreamer使用类型系统来确保元素之间传递的数据采用可识别的格式。类型系统对于确保在链接元素之间的cap时完全指定格式所需的参数正确匹配也很重要。元素之间建立的每个链接都具有指定的类型和可选的一组属性。查看更多关于caps协商上限的谈判。
GStreamer uses a type system to ensure that the data passed between elements is in a recognized format. The type system is also important for ensuring that the parameters required to fully specify a format match up correctly when linking pads between elements. Each link that is made between elements has a specified type and optionally a set of properties. See more about caps negotiation in Caps negotiation.
The Basic Types
GStreamer已经支持许多基本媒体类型。以下是GStreamer中用于缓冲区的一些基本类型的表格。该表包含名称（“媒体类型”）以及类型的描述，与类型关联的属性以及每个属性的含义。已定义类型列表中包含受支持类型的完整列表。
GStreamer already supports many basic media types. Following is a table of a few of the basic types used for buffers in GStreamer. The table contains the name ("media type") and a description of the type, the properties associated with the type, and the meaning of each property. A full list of supported types is included in List of Defined Types.
| Media Type | Description | Property | Property Type | Property Values | Property Description |
|---|---|---|---|---|---|
| audio/* | All audio types | rate | integer | greater than 0 | The sample rate of the data, in samples (per channel) per second. |
| channels | integer | greater than 0 | The number of channels of audio data. | ||
| audio/x-raw | Unstructured and uncompressed raw integer audio data. | format | string | S8 U8 S16LE S16BE U16LE U16BE S24_32LE S24_32BE U24_32LE U24_32BE S32LE S32BE U32LE U32BE S24LE S24BE U24LE U24BE S20LE S20BE U20LE U20BE S18LE S18BE U18LE U18BE F32LE F32BE F64LE F64BE | The format of the sample data. |
| audio/mpeg | Audio data compressed using the MPEG audio encoding scheme. | mpegversion | integer | 1, 2 or 4 | The MPEG-version used for encoding the data. The value 1 refers to MPEG-1, -2 and -2.5 layer 1, 2 or 3. The values 2 and 4 refer to the MPEG-AAC audio encoding schemes. |
| framed | boolean | 0 or 1 | A true value indicates that each buffer contains exactly one frame. A false value indicates that frames and buffers do not necessarily match up. | ||
| layer | integer | 1, 2, or 3 | The compression scheme layer used to compress the data (only if mpegversion=1). | ||
| bitrate | integer | greater than 0 | The bitrate, in bits per second. For VBR (variable bitrate) MPEG data, this is the average bitrate. | ||
| audio/x-vorbis | Vorbis audio data | There are currently no specific properties defined for this type. |