gstream学习4-Time management

官网：https://gstreamer.freedesktop.org/

Demo基础教程：hhttps://gstreamer.freedesktop.org/documentation/tutorials/basic/concepts.html

Demo下载地址：git://anongit.freedesktop.org/gstreamer/gst-docs

Goal

学习时间相关的内容，尤其是时间的查询以及Seek功能。

This tutorial shows how to use GStreamer time-related facilities(工具). In particular(尤其):

• How to query(查询) the pipeline for information like stream position or duration.
• How to seek (jump) to a different position (time) inside the stream.

Introduction

GstQuery 可以获取bin或者pad的一条消息，比如是否允许Seek等。通过定期调用GstQuery 获取进度，可以实时显示进度

GstQuery is a mechanism that allows asking an element or pad for a piece of information. In this example we ask the pipeline if seeking is allowed (some sources, like live streams, do not allow seeking). If it is allowed, then, once the movie has been running for ten seconds, we skip to a different position using a seek.

In the previous tutorials, once we had the pipeline setup and running, our main function just sat(sit过去式，坐着) and waited to receive an ERROR or an EOS through the bus. Here, we modify this function to periodically(定期) wake up and query the pipeline for the stream position, so we can print it on the screen. This is similar to what a media player would do, updating the user Interface on a periodic(周期性的) basis.

Finally, the stream duration is queried and updated whenever it changes.

Seeking example

Copy this code into a text file named basic-tutorial-4.c (or find it in your GStreamer installation).

basic-tutorial-4.c

#include <gst/gst.h>

/* Structure to contain all our information, so we can pass it around */
typedef struct _CustomData {
  GstElement *playbin;  /* Our one and only element */
  gboolean playing;      /* Are we in the PLAYING state? */
  gboolean terminate;    /* Should we terminate execution? */
  gboolean seek_enabled; /* Is seeking enabled for this media? */
  gboolean seek_done;    /* Have we performed the seek already? */
  gint64 duration;       /* How long does this media last, in nanoseconds */
} CustomData;

/* Forward definition of the message processing function */
static void handle_message (CustomData *data, GstMessage *msg);

int main(int argc, char *argv[]) {
  CustomData data;
  GstBus *bus;
  GstMessage *msg;
  GstStateChangeReturn ret;

  data.playing = FALSE;
  data.terminate = FALSE;
  data.seek_enabled = FALSE;
  data.seek_done = FALSE;
  data.duration = GST_CLOCK_TIME_NONE;

  /* Initialize GStreamer */
  gst_init (&argc, &argv);

  /* Create the elements */
  data.playbin = gst_element_factory_make ("playbin", "playbin");

  if (!data.playbin) {
    g_printerr ("Not all elements could be created.\n");
    return -1;
  }

  /* Set the URI to play */
  g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);

  /* Start playing */
  ret = gst_element_set_state (data.playbin, GST_STATE_PLAYING);
  if (ret == GST_STATE_CHANGE_FAILURE) {
    g_printerr ("Unable to set the pipeline to the playing state.\n");
    gst_object_unref (data.playbin);
    return -1;
  }

  /* Listen to the bus */
  bus = gst_element_get_bus (data.playbin);
  do {
    msg = gst_bus_timed_pop_filtered (bus, 100 * GST_MSECOND,
        GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_ERROR | GST_MESSAGE_EOS | GST_MESSAGE_DURATION);

    /* Parse message */
    if (msg != NULL) {
      handle_message (&data, msg);
    } else {
      /* We got no message, this means the timeout expired */
      if (data.playing) {
        gint64 current = -1;

        /* Query the current position of the stream */
        if (!gst_element_query_position (data.playbin, GST_FORMAT_TIME, &current)) {
          g_printerr ("Could not query current position.\n");
        }

        /* If we didn't know it yet, query the stream duration */
        if (!GST_CLOCK_TIME_IS_VALID (data.duration)) {
          if (!gst_element_query_duration (data.playbin, GST_FORMAT_TIME, &data.duration)) {
            g_printerr ("Could not query current duration.\n");
          }
        }

        /* Print current position and total duration */
        g_print ("Position %" GST_TIME_FORMAT " / %" GST_TIME_FORMAT "\r",
            GST_TIME_ARGS (current), GST_TIME_ARGS (data.duration));

        /* If seeking is enabled, we have not done it yet, and the time is right, seek */
        if (data.seek_enabled && !data.seek_done && current > 10 * GST_SECOND) {
          g_print ("\nReached 10s, performing seek...\n");
          gst_element_seek_simple (data.playbin, GST_FORMAT_TIME,
              GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT, 30 * GST_SECOND);
          data.seek_done = TRUE;
        }
      }
    }
  } while (!data.terminate);

  /* Free resources */
  gst_object_unref (bus);
  gst_element_set_state (data.playbin, GST_STATE_NULL);
  gst_object_unref (data.playbin);
  return 0;
}

static void handle_message (CustomData *data, GstMessage *msg) {
  GError *err;
  gchar *debug_info;

  switch (GST_MESSAGE_TYPE (msg)) {
    case GST_MESSAGE_ERROR:
      gst_message_parse_error (msg, &err, &debug_info);
      g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
      g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
      g_clear_error (&err);
      g_free (debug_info);
      data->terminate = TRUE;
      break;
    case GST_MESSAGE_EOS:
      g_print ("End-Of-Stream reached.\n");
      data->terminate = TRUE;
      break;
    case GST_MESSAGE_DURATION:
      /* The duration has changed, mark the current one as invalid */
      data->duration = GST_CLOCK_TIME_NONE;
      break;
    case GST_MESSAGE_STATE_CHANGED: {
      GstState old_state, new_state, pending_state;
      gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
      if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->playbin)) {
        g_print ("Pipeline state changed from %s to %s:\n",
            gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));

        /* Remember whether we are in the PLAYING state or not */
        data->playing = (new_state == GST_STATE_PLAYING);

        if (data->playing) {
          /* We just moved to PLAYING. Check if seeking is possible */
          GstQuery *query;
          gint64 start, end;
          query = gst_query_new_seeking (GST_FORMAT_TIME);
          if (gst_element_query (data->playbin, query)) {
            gst_query_parse_seeking (query, NULL, &data->seek_enabled, &start, &end);
            if (data->seek_enabled) {
              g_print ("Seeking is ENABLED from %" GST_TIME_FORMAT " to %" GST_TIME_FORMAT "\n",
                  GST_TIME_ARGS (start), GST_TIME_ARGS (end));
            } else {
              g_print ("Seeking is DISABLED for this stream.\n");
            }
          }
          else {
            g_printerr ("Seeking query failed.");
          }
          gst_query_unref (query);
        }
      }
    } break;
    default:
      /* We should not reach here */
      g_printerr ("Unexpected message received.\n");
      break;
  }
  gst_message_unref (msg);
}

Walkthrough

/* Structure to contain all our information, so we can pass it around */
typedef struct _CustomData {
  GstElement *playbin;  /* Our one and only element */
  gboolean playing;      /* Are we in the PLAYING state? */
  gboolean terminate;    /* Should we terminate execution? */
  gboolean seek_enabled; /* Is seeking enabled for this media? */
  gboolean seek_done;    /* Have we performed the seek already? */
  gint64 duration;       /* How long does this media last, in nanoseconds */
} CustomData;

/* Forward definition of the message processing function */
static void handle_message (CustomData *data, GstMessage *msg);

组合一个结构体，这里有一个特殊的element-playbin,既有source又有sink，这个结构体将会被传输到自己的消息处理函数handle_message.

We start by defining a structure to contain all our information, so we can pass it around to other functions. In particular, in this example we move the message handling code to its own function handle_message because it is growing a bit too big.

We then build a pipeline composed of(组成) a single element, a playbin, which we already saw in Basic tutorial 1: Hello world!. However, playbin is in itself a pipeline, and in this case it is the only element in the pipeline, so we directly use the playbin element. We will skip the details: the URI of the clip is given to playbin via the URI property and the pipeline is set to the playing state.

等待pipeline的结果。这里出去状态改变、error、EOS还增加了总时长发生变化的通知。第二个参数也不是无限等待，而是设置定时触发。gstreamr时间参数都是以纳秒为单位，所以需要乘GST_SECOND 或 GST_MSECOND字样来表示秒或毫秒。

msg = gst_bus_timed_pop_filtered (bus, 100 * GST_MSECOND,
    GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_ERROR | GST_MESSAGE_EOS | GST_MESSAGE_DURATION);

Previously we did not provide a timeout to gst_bus_timed_pop_filtered(), meaning that it didn't return until a message was received. Now we use a timeout of 100 milliseconds, so, if no message is received during one tenth of a second, the function will return NULL. We are going to use this logic to update our “UI”.

Note that the desired timeout must be specified as a GstClockTime, hence, in nanoseconds. Numbers expressing different time units then, should be multiplied by macros like GST_SECOND or GST_MSECOND. This also makes your code more readable.

If we got a message, we process it in the handle_message function (next subsection), otherwise:

User interface refreshing

/* We got no message, this means the timeout expired */
if (data.playing) {

If the pipeline is in PLAYING state, it is time to refresh the screen. We don't want to do anything if we are not in PLAYING state, because most queries would fail.

We get here approximately 10 times per second, a good enough refresh rate for our UI. We are going to print on screen the current media position, which we can learn by querying the pipeline. This involves a few steps that will be shown in the next subsection, but, since position and duration are common enough queries, GstElement offers easier(更容易的), ready-made(现成的) alternatives(替代):

/* Query the current position of the stream */
if (!gst_element_query_position (data.pipeline, GST_FORMAT_TIME, &current)) {
  g_printerr ("Could not query current position.\n");
}

gst_element_query_position() hides the management of the query object and directly provides us with the result.

如果总时长没有获取，执行获取总时长函数，因为之前监听了总时长事件，所以在hand_message中可以获取到总时长。

/* If we didn't know it yet, query the stream duration */
if (!GST_CLOCK_TIME_IS_VALID (data.duration)) {
  if (!gst_element_query_duration (data.pipeline, GST_FORMAT_TIME, &data.duration)) {
     g_printerr ("Could not query current duration.\n");
  }
}

Now is a good moment to know the length of the stream, with another GstElement helper function: gst_element_query_duration()

/* Print current position and total duration */
g_print ("Position %" GST_TIME_FORMAT " / %" GST_TIME_FORMAT "\r",
    GST_TIME_ARGS (current), GST_TIME_ARGS (data.duration));

Note the usage of the GST_TIME_FORMAT and GST_TIME_ARGS macros to provide a user-friendly representation of GStreamer times.

当运行时长大于10秒时，触发Seek操作，调用 gst_element_seek_simple函数。 gst_element_seek_simple第一个参数是对应的pipeline，这里就是playbin(element).第二个参数是时间类型GST_FORMAT_TIME(表示微妙)，第三个参数GstSeekFlags，第四个参数要Seek的位置。其中第三个参数常用的有：GST_SEEK_FLAG_FLUSH(清空管道中的数据)，GST_SEEK_FLAG_KEY_UNIT(是否从关键帧开始)，GST_SEEK_FLAG_ACCURATE(精准Seek，一般gstream通过估算Seek的时间，可以很快的定位到需要Seek的位置，但是由于某些文件或者结构没有足够的Seek索引，导致Seek的位置不准确，可以通过设置这个参数来达到准确的Seek位置，当然这可能会消耗很长时间)。

/* If seeking is enabled, we have not done it yet, and the time is right, seek */
if (data.seek_enabled && !data.seek_done && current > 10 * GST_SECOND) {
  g_print ("\nReached 10s, performing seek...\n");
  gst_element_seek_simple (data.playbin, GST_FORMAT_TIME,
      GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT, 30 * GST_SECOND);
  data.seek_done = TRUE;
}

Now we perform the seek, “simply” by calling gst_element_seek_simple() on the pipeline. A lot of the intricacies(复杂的) of seeking are hidden in this method, which is a good thing!

Let's review the parameters:

GST_FORMAT_TIME indicates that we are specifying the destination in time units. Other seek-formats use different units.

Then come the GstSeekFlags, let's review the most common:

GST_SEEK_FLAG_FLUSH: This discards all data currently in the pipeline before doing the seek. Might pause a bit while the pipeline is refilled and the new data starts to show up, but greatly increases the “responsiveness” of the application. If this flag is not provided, “stale” data might be shown for a while until the new position appears at the end of the pipeline.

GST_SEEK_FLAG_KEY_UNIT: With most encoded video streams, seeking to arbitrary(任意的) positions is not possible but only to certain frames called Key Frames. When this flag is used, the seek will actually move to the closest key frame and start producing data straight away. If this flag is not used, the pipeline will move internally to the closest key frame (it has no other alternative) but data will not be shown until it reaches the requested position. This last alternative is more accurate(精准的), but might take longer.

GST_SEEK_FLAG_ACCURATE: Some media clips do not provide enough indexing information, meaning that seeking to arbitrary positions is time-consuming. In these cases, GStreamer usually estimates the position to seek to, and usually works just fine. If this precision is not good enough for your case (you see seeks not going to the exact time you asked for), then provide this flag. Be warned that it might take longer to calculate the seeking position (very long, on some files).

Finally, we provide the position to seek to. Since we asked for GST_FORMAT_TIME, the value must be in nanoseconds so we express the time in seconds, for simplicity, and then multiply by GST_SECOND.

Message Pump

handle_message处理所有的消息，报考获取总时长，状态改变等。

The handle_message function processes all messages received through the pipeline's bus. ERROR and EOS handling is the same as in previous tutorials, so we skip to the interesting part:

case GST_MESSAGE_DURATION:
  /* The duration has changed, mark the current one as invalid */
  data->duration = GST_CLOCK_TIME_NONE;
  break;

This message is posted on the bus whenever the duration of the stream changes. Here we simply mark the current duration as invalid, so it gets re-queried later.

case GST_MESSAGE_STATE_CHANGED: {
  GstState old_state, new_state, pending_state;
  gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
  if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->pipeline)) {
    g_print ("Pipeline state changed from %s to %s:\n",
        gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));

    /* Remember whether we are in the PLAYING state or not */
    data->playing = (new_state == GST_STATE_PLAYING);

seek和时间查询仅仅在PAUSED和PLAYING状态进行。第一次进入PLAYING状态的时候，我们询问是否支持Seek以及可以Seek的范围。步骤是new一个请求query(gst_query_new_seeking )，然后获取请求(gst_element_query)，然后解析请求结果(gst_query_parse_seeking)，当然别忘记释放。

Seeks and time queries generally only get a valid reply when in the PAUSED or PLAYING state, since all elements have had a chance to receive information and configure themselves. Here, we use the playing variable to keep track of whether the pipeline is in PLAYING state. Also, if we have just entered the PLAYING state, we do our first query. We ask the pipeline if seeking is allowed on this stream:

if (data->playing) {
  /* We just moved to PLAYING. Check if seeking is possible */
  GstQuery *query;
  gint64 start, end;
  query = gst_query_new_seeking (GST_FORMAT_TIME);
  if (gst_element_query (data->pipeline, query)) {
    gst_query_parse_seeking (query, NULL, &data->seek_enabled, &start, &end);
    if (data->seek_enabled) {
      g_print ("Seeking is ENABLED from %" GST_TIME_FORMAT " to %" GST_TIME_FORMAT "\n",
          GST_TIME_ARGS (start), GST_TIME_ARGS (end));
    } else {
      g_print ("Seeking is DISABLED for this stream.\n");
    }
  }
  else {
    g_printerr ("Seeking query failed.");
  }
  gst_query_unref (query);
}

gst_query_new_seeking() creates a new query object of the "seeking" type, with GST_FORMAT_TIME format. This indicates that we are interested in seeking by specifying the new time to which we want to move. We could also ask for GST_FORMAT_BYTES, and then seek to a particular byte position inside the source file, but this is normally less useful.

This query object is then passed to the pipeline with gst_element_query(). The result is stored in the same query, and can be easily retrieved with gst_query_parse_seeking(). It extracts a boolean indicating if seeking is allowed, and the range in which seeking is possible.

Don't forget to unref the query object when you are done with it.

And that's it! With this knowledge a media player can be built which periodically(周期性的) updates a slider based on the current stream position and allows seeking by moving the slider!

Conclusion

This tutorial has shown:

• How to query the pipeline for information using GstQuery

• How to obtain common information like position and duration using gst_element_query_position() and gst_element_query_duration()

• How to seek to an arbitrary position in the stream using gst_element_seek_simple()

• In which states all these operations can be performed.

The next tutorial shows how to integrate(整合) GStreamer with a Graphical User Interface toolkit.

Remember that attached to this page you should find the complete source code of the tutorial and any accessory(附属的) files needed to build it.

It has been a pleasure having you here, and see you soon!