Introduction to ogg file packaging format

Core reference xiph official website

ogg data structure

datatype	purpose
ogg_page	This structure encapsulates data into one ogg bitstream page. The page information during encoding is output here
ogg_stream_state	This structure contains current encode/decode data for a logical bitstream. 代表当前流
ogg_packet	This structure encapsulates the data and metadata for a single Ogg packet.
ogg_sync_state	Contains bitstream synchronization information.

The largest structure in the above ogg is the ogg stream. Compared with the storage structure of ogg, ogg_page is used for data paging, but for users, ogg_packet is used to load and parse data.

Coding related

Overview

When encoding, the encoding engine will output raw packets which must be placed into an Ogg bitstream. Raw packets are inserted into the stream, and an ogg_page is output when enough packets have been written to create a full page. The pages output are pointers to buffered packet segments, and can then be written out and saved as an ogg stream. There are a couple of basic steps: Use the encoding engine to produce a raw packet of data. Call ogg_stream_packetin to submit a raw packet to the stream. Use ogg_stream_pageout to output a page, if enough data has been submitted. Otherwise, continue submitting data.
It means that the original data should be encapsulated in ogg_packet and written to ogg_stream_state through the ogg_stream_packetin method. If the packet is enough, the data should be written in ogg_page through the ogg_stream_pageout method.

function	purpose
ogg_stream_packetin	Submits a raw packet to the streaming layer, so that it can be formed into a page.
ogg_stream_iovecin	iovec version of ogg_stream_packetin() above.
ogg_stream_pageout	Outputs a completed page if the stream contains enough packets to form a full page. The return value is not 0 code output is successful
ogg_stream_pageout_fill	Similar to ogg_stream_pageout(), but specifies a page spill threshold in bytes.
ogg_stream_flush	Forces any remaining packets in the stream to be returned as a page of any size.
ogg_stream_flush_fill	Similar to ogg_stream_flush(), but specifies a page spill threshold in bytes.

Create and destroy

Create
ogg_stream_init(&stream_, 0 /* serial number */);
destroy
ogg_stream_clear(&stream_);

Encoding data package

typedef struct {
    
    
  unsigned char *packet;//数据
  long  bytes;//数据大小
  long  b_o_s;//1代表开始包
  long  e_o_s;//1代表结束包
  //A number indicating the position of this packet in the decoded data. This is the last sample, frame or other unit of information ('granule') that can be completely decoded from this packet.
  ogg_int64_t  granulepos;
  ogg_int64_t  packetno;//Sequential number of this packet in the ogg bitstream.

} ogg_packet;

Single stream data write

// Write the most recent buffer of Opus data into an Ogg packet.
  ogg_packet frame_packet;
  frame_packet.b_o_s = 0;
  frame_packet.e_o_s = flush ? 1 : 0;
  // According to
  // https://tools.ietf.org/html/draft-ietf-codec-oggopus-14#section-4 the
  // granule position should include all samples up to the last packet completed
  // on the page, so we need to update granule_position_ before assigning it to
  // the packet.  If we're closing the stream, we don't assume that the last
  // packet includes a full frame.
  if (flush) {
    
    
    granule_position_ += (elements_in_pcm_frame_ / num_channels_);
  } else {
    
    
    granule_position_ += frame_size_;
  }
  frame_packet.granulepos = granule_position_;
  frame_packet.packetno = packet_count_;
  frame_packet.packet = opus_frame_bytes;
  frame_packet.bytes = opus_bytes_length;
  // Add the data packet into the stream.
  packet_count_++;
  ogg_stream_packetin(&stream_, &frame_packet);

Get output data

void OggOpusEncoder::AppendOggStateToBuffer(std::vector<unsigned char>* buffer,
                                            bool flush_ogg_stream) {
    
    
  int (*write_fun)(ogg_stream_state*, ogg_page*) =
      flush_ogg_stream ? &ogg_stream_flush : &ogg_stream_pageout;
  while (write_fun(&stream_, &page_) != 0) {
    
    
    const int initial_size = buffer->size();
    buffer->resize(buffer->size() + page_.header_len + page_.body_len);
    memcpy(buffer->data() + initial_size, page_.header, page_.header_len);
    memcpy(buffer->data() + initial_size + page_.header_len, page_.body,
           page_.body_len);
  }
}

Decoding related

basic method

function	use
ogg_sync_pageseek	Finds the borders of pages and resynchronizes the stream. -n means that we skipped n bytes within the bitstream.0 means that the page isn’t ready and we need more data, or than an internal error occurred. No bytes have been skipped. n means that the page was synced at the current location, with a page length of n bytes.
ogg_sync_buffer	Exposes a buffer from the synchronization layer in order to read data. The return buffer is introduced to prepare for the next data input
ogg_sync_wrote	Tells the synchronization layer how many bytes were written into the buffer.
ogg_stream_pagein	Submits a complete page to the stream layer. Submit the page information to the stream layer
ogg_stream_packetout	Outputs a packet to the codec-specific decoding engine. Take out the packet information and process the original data

Create and destroy

Create
ogg_sync_state ogsync; ogg_sync_init(&ogsync);
destroy
ogg_sync_clear

Traverse processing data

Processing structure

ogg_sync_init(&ogsync);
while(get_next_page(file, &ogsync, &page, &written)) {
    
    
    ...
    p->process_page(p, &page);
    ...
    
}
ogg_sync_clear(&ogsync)

Write data to synchronizer

while((ret = ogg_sync_pageseek(ogsync, page)) <= 0) {
    
    
    if(ret < 0) {
    
    
        /* unsynced, we jump over bytes to a possible capture - we don't need to read more just yet */
        oi_warn(_("WARNING: Hole in data (%d bytes) found at approximate offset %" I64FORMAT " bytes. Corrupted Ogg.\n"), -ret, *written);
        continue;
    }

    /* zero return, we didn't have enough data to find a whole page, read */
    buffer = ogg_sync_buffer(ogsync, CHUNK);
    bytes = fread(buffer, 1, CHUNK, f);
    if(bytes <= 0) {
    
    
        ogg_sync_wrote(ogsync, 0);
        return 0;
    }
    ogg_sync_wrote(ogsync, bytes);
    *written += bytes;
}

File header processing example

//初始化流
ogg_stream_init(&stream->os, serial);
//ogg_stream_init
ogg_stream_pagein(&stream->os, page);
res = ogg_stream_packetout(&stream->os, &packet);
if(res <= 0) {
    
    
    oi_warn(_("WARNING: Invalid header page, no packet found\n"));
    null_start(stream);
}
else if(packet.bytes >= 19 && memcmp(packet.packet, "OpusHead", 8)==0)
...

/* re-init, ready for processing */
ogg_stream_clear(&stream->os);
ogg_stream_init(&stream->os, serial);
//此处是为了下次重新解析头

Audio data analysis

The analysis of audio data is also parsed for each page according to the ogg format