Analysis of Haikang and National Standard PS Stream Data

    A recent project that needs to be connected to the video data transmitted by the Haikang camera. After analysis, it is known that the transmitted data is PS encapsulated data and cannot be played directly. The PS stream needs to be parsed into a h264 raw stream. After research, there are two methods: 1. Directly throw it to ffmpeg for parsing, this method needs to use AVIOContext to customize the input; 2. Parse the PS stream by yourself, this method is more straightforward, as long as you are familiar with the PS parsing process, you can do it. I realized both methods. At first, I used ffmpeg. Later, I found out that because of the need to open threads, asynchronous reading and writing, the complexity was high, and then I wondered if I could parse the PS by myself, so I found a lot of information on the Internet, and finally realized it. , Parsing PS by yourself is relatively simple, no need to open threads, no asynchronous processing, just call an algorithm to get the parsing result. The realization of the two methods are recorded here, and I hope it will be helpful to the latecomers.

 1. Use ffmpeg to parse the PS stream

First, you need to customize the input with AVIOContext, I encapsulated it as a H264Reader class, as follows:

#pragma once
#include <stdio.h>
#include <thread>
extern "C" {
#include "libavformat/avformat.h"
#include "libavcodec/avcodec.h"
}
#pragma comment(lib,"avformat.lib")
#pragma comment(lib,"avutil.lib")
#pragma comment(lib,"avcodec.lib")
#include "utils.h"

class H264Reader
{
public:
	H264Reader(DevStreamHandle* aStreamHandle);

	int Run(DevStreamHandle* aStreamHandle);
	virtual ~H264Reader();
	void Stop() {
		this->running = false;
	}
private:

	AVFormatContext *ic = NULL;
	AVIOContext *avio = NULL;
	unsigned char *avio_ctx_buffer = NULL;

public:
	bool running = true;

};

#include "H264Reader.h"
#include "BlockingRingBuffer.h"
#include <iostream>
using namespace std;

H264Reader::H264Reader(DevStreamHandle* aStreamHandle)
{
	std::thread([&](DevStreamHandle *handle) {
		Run(handle);
	}, aStreamHandle).detach();
}

static int read_packet(void *opaque, uint8_t *buf, int buf_size) {
	DevStreamHandle *handle = (DevStreamHandle*)opaque;
	En_BlockingRingBuffer_FetchResult ret = handle->ringBuff->Fetch(buf, buf_size); //
	
	if (ret == RBFR_OK)
		return buf_size;
	else {
		return 0;
	}
}

int H264Reader::Run(DevStreamHandle* stream_handle)
{
	int isopen = -1;
	size_t avio_ctx_buffer_size = 1024*32;// 
	ic = avformat_alloc_context();
	avio_ctx_buffer = (unsigned char *)av_malloc(avio_ctx_buffer_size);
	avio = avio_alloc_context(avio_ctx_buffer, avio_ctx_buffer_size, 0, stream_handle, read_packet, NULL, NULL);
	ic->pb = avio;
	isopen = avformat_open_input(&ic, NULL, NULL, NULL);
	if (isopen < 0) {
		printf("Could not open input----->>>\n");
		return -1;
	}
	AVPacket *pkt = av_packet_alloc();
	while (running)
	{
		int re = av_read_frame(ic, pkt);
		if (re != 0){
			av_packet_unref(pkt);
			printf("~~~~~~~~~av_read_frame()= %d\n", re);
			continue;
		}
		//这里就把h264裸流数据投送给需要的地方
		//PostData((char*)pkt->data, pkt->size, stream_handle);
		av_packet_unref(pkt);
	}
	if (avio) {
		av_freep(&avio->buffer);
		av_freep(&avio);
	}
	return 0;
}

H264Reader::~H264Reader()
{
	//av_free(avio_ctx_buffer);
}

 

The H264Reader class starts a thread and calls the Run() function of H264Reader in the thread. To use AVIOContext, you need to define a read callback function. The above code avio = avio_alloc_context(avio_ctx_buffer, avio_ctx_buffer_size, 0, this, read_packet, NULL, NULL) This line binds the custom callback function read_packet to AVIOContext, where the fourth parameter is User data, where the DevStreamHandle structure type stream_handle is passed in, and this variable can be obtained in the callback. Write the PS stream to ffmpeg in the callback. ffmpeg will continuously call this callback, and then continuously call av_read_frame(ic, pkt) in the loop. The returned AVPacket variable pkt is the h264 raw stream that has been parsed by ffmpeg.

The difficulty of using AVIOContext custom input is that you need to design a buffer, the writing thread needs to write data of variable size to the buffer, and the reading thread needs to read data of variable size to the buffer, because the writing and reading The size is not fixed, so a fixed-size buffer cannot be defined. Here I use a ring blocking buffer list BlockingRingBuffer.

In the writing thread, you need to call the Cat() function of BlockingRingBuffer to write to the PS stream:

// 取到PS流，存储在ringBuff
int  FacdNetMediaServer::cbPreStreamMedia(HSTREAM hStream, const StreamMediaFrame *cFrame, DWORD dwUserData)
{
	DevStreamHandle *pHandle = (DevStreamHandle*)dwUserData;
	if (pHandle != NULL && pHandle->ringBuff != NULL){
		pHandle->ringBuff->Cat((const unsigned char*)cFrame->cFrameBuffer.pBuffer, cFrame->cFrameBuffer.dwBufLen);
	}
	return 0;
}

 Call the Fecth() function of BlockingRingBuffer in the read callback of AVIOContext to read the PS stream:

static int read_packet(void *opaque, uint8_t *buf, int buf_size) {
	H264Reader *h264 = (H264Reader*)opaque;
	DevStreamHandle *handle = h264->streamHandle;
	En_BlockingRingBuffer_FetchResult ret =handle->ringBuff->Fetch(buf, buf_size); //
	
	if (ret == RBFR_OK)
		return buf_size;
	else {
		return 0;
	}
}

Second, use your own method of parsing the PS stream

This method is much simpler. The main idea is that the packets between the two PS stream headers are spliced ​​into a data packet and thrown to the PS parsing function. The result is an h264 frame. The PS header is identified by 0x000001ba, so the splicing code is as follows :

#define MAX_FRAME_SIZE 1024*1024*5
char *h264_pkt=new char[MAX_FRAME_SIZE];
int pkt_len;
bool isfirst = true;
int  cbRealStream(char* ps_buffer,int len)
{
	if (ps_buffer != NULL && len>0){
		char *h264Frame;
		int h264Len;
		
		//printf("%02x %02x %02x %02x\n", ps_buffer[0], ps_buffer[1], ps_buffer[2], ps_buffer[3]);
		//查找ps头 0x000001BA
		if (ps_buffer[0] == '\x00' && ps_buffer[1] == '\x00' && ps_buffer[2] == '\x01' && ps_buffer[3] == '\xba')
		{
			if (!isfirst)
			{
				//此包为ps新的一帧,每次到这里都先处理存储好的前一帧
				GetH246FromPs(ps_pkt, pkt_len, &h264Frame, &h264Len);
				//海康流特殊处理部分:分界符数据(nal_unit_type=9)或补充增强信息单元(nal_unit_type=6)，如果直接送入解码器，有可能会出现问题,直接舍弃.00 00 01 bd和 00 00 01 c0为私有标志和音频数据舍弃
				if (h264Frame[0] >> 5 == '\x06' || h264Frame[0] >> 5 == '\x09' || h264Frame[0] >> 5 == '\x0a' || h264Frame[0] >> 5 == '\x0b' || h264Frame[0] >> 5 == '\x0c')
				{
				}
				else {//这就是获取的含有标准正常H264数据的帧 我们开始进行处理,将该帧存入Deque,该deque只负责存储所有的一帧帧的数据
					bool ret = pHandle->cam->deliverFrame(h264Frame, h264Len);
					
				}
			}
			//各个变量初始化,开始拼接下一个帧(可能收到的N个包才能组成一个帧,所以这里有一个拼接操作,主要是一个帧的头部指针一直memcpy,把内存向后叠加)
			pkt_len = 0;
			memset(ps_pkt, 0, MAX_FRAME_SIZE);
			memcpy(ps_pkt + pkt_len, ps_buffer, len);
			pkt_len += len;
			isfirst = false;
		}
		else {//当然如果开头不是0x000001BA,默认为一个帧的中间部分,我们将这部分内存顺着帧的开头向后存储
			if (!isfirst)
			{
				//排除音频和私有数据
				if (ps_buffer[0] == '\x00' && ps_buffer[1] == '\x00' && ps_buffer[2] == '\x01' && (ps_buffer[3] == '\xc0' || ps_buffer[3] == '\xbd'))
				{
					return 0;
				}
				//这是正常的帧数据
				if (pkt_len + len > MAX_FRAME_SIZE) {
					LogError("帧大小超过5MB");
					return -1;
				}
				memcpy(ps_pkt + pkt_len, ps_buffer, len);
				pkt_len += len;
			}
		}
		
	}
	return 0;
}

After splicing the PS frames, call the PS parsing function GetH246FromPs() to get the h264 data. The PS parsing function is defined as follows:

#pragma once
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#pragma pack(1)

union littel_endian_size
{
	unsigned short int	length;
	unsigned char		byte[2];
};

struct pack_start_code
{
	unsigned char start_code[3];
	unsigned char stream_id[1];
};

struct program_stream_pack_header
{
	pack_start_code PackStart;// 4
	unsigned char Buf[9];
	unsigned char stuffinglen;
};

struct program_stream_map
{
	pack_start_code PackStart;
	littel_endian_size PackLength;//we mast do exchange
								  //program_stream_info_length
								  //info
								  //elementary_stream_map_length
								  //elem
};

struct program_stream_e
{
	pack_start_code		PackStart;
	littel_endian_size	PackLength;//we mast do exchange
	char				PackInfo1[2];
	unsigned char		stuffing_length;
};

#pragma pack()

int inline ProgramStreamPackHeader(char* Pack, int length, char **NextPack, int *leftlength)
{
	//printf("[%s]%x %x %x %x\n", __FUNCTION__, Pack[0], Pack[1], Pack[2], Pack[3]);
	//通过 00 00 01 ba头的第14个字节的最后3位来确定头部填充了多少字节
	program_stream_pack_header *PsHead = (program_stream_pack_header *)Pack;
	unsigned char pack_stuffing_length = PsHead->stuffinglen & '\x07';

	*leftlength = length - sizeof(program_stream_pack_header) - pack_stuffing_length;//减去头和填充的字节
	*NextPack = Pack + sizeof(program_stream_pack_header) + pack_stuffing_length;

	if (*leftlength<4) return 0;

	//printf("[%s]2 %x %x %x %x\n", __FUNCTION__, (*NextPack)[0], (*NextPack)[1], (*NextPack)[2], (*NextPack)[3]);

	return *leftlength;
}

inline int ProgramStreamMap(char* Pack, int length, char **NextPack, int *leftlength, char **PayloadData, int *PayloadDataLen)
{
	//printf("[%s]%x %x %x %x\n", __FUNCTION__, Pack[0], Pack[1], Pack[2], Pack[3]);

	program_stream_map* PSMPack = (program_stream_map*)Pack;

	//no payload
	*PayloadData = 0;
	*PayloadDataLen = 0;

	if (length < sizeof(program_stream_map)) return 0;

	littel_endian_size psm_length;
	psm_length.byte[0] = PSMPack->PackLength.byte[1];
	psm_length.byte[1] = PSMPack->PackLength.byte[0];

	*leftlength = length - psm_length.length - sizeof(program_stream_map);

	//printf("[%s]leftlength %d\n", __FUNCTION__, *leftlength);

	if (*leftlength <= 0) return 0;

	*NextPack = Pack + psm_length.length + sizeof(program_stream_map);

	return *leftlength;
}

inline int Pes(char* Pack, int length, char **NextPack, int *leftlength, char **PayloadData, int *PayloadDataLen)
{
	//printf("[%s]%x %x %x %x\n", __FUNCTION__, Pack[0], Pack[1], Pack[2], Pack[3]);
	program_stream_e* PSEPack = (program_stream_e*)Pack;

	*PayloadData = 0;
	*PayloadDataLen = 0;

	if (length < sizeof(program_stream_e)) return 0;

	littel_endian_size pse_length;
	pse_length.byte[0] = PSEPack->PackLength.byte[1];
	pse_length.byte[1] = PSEPack->PackLength.byte[0];
	if (pse_length.length + 6 > length)
	{
		pse_length.length = length - 6;
		/*int PES_packet_length_fixed = length - 6;
		Pack[4] = (char)((PES_packet_length_fixed && 0xFF00) >> 8);
		Pack[5] = (char)(PES_packet_length_fixed & 0x00FF);*/
	}

	*PayloadDataLen = pse_length.length - 2 - 1 - PSEPack->stuffing_length;
	if (*PayloadDataLen>0)
		*PayloadData = Pack + sizeof(program_stream_e) + PSEPack->stuffing_length;

	*leftlength = length - pse_length.length - sizeof(pack_start_code) - sizeof(littel_endian_size);

	//printf("[%s]leftlength %d\n", __FUNCTION__, *leftlength);

	if (*leftlength <= 0) return 0;

	*NextPack = Pack + sizeof(pack_start_code) + sizeof(littel_endian_size) + pse_length.length;

	return *leftlength;
}

int inline GetH246FromPs(char* buffer, int length, char **h264Buffer, int *h264length)
{
	int leftlength = 0;
	char *NextPack = buffer;

	*h264Buffer = buffer;
	*h264length = 0;

	leftlength = length;

	if (ProgramStreamPackHeader(buffer, length, &NextPack, &leftlength) == 0)
		return 0;

	char *PayloadData = NULL;
	int PayloadDataLen = 0;

	while (leftlength >= sizeof(pack_start_code))
	{
		PayloadData = NULL;
		PayloadDataLen = 0;

		
		if (NextPack
			&& NextPack[0] == '\x00'
			&& NextPack[1] == '\x00'
			&& NextPack[2] == '\x01'
			&& NextPack[3] == '\xE0')
		{
			//接着就是流包，说明是非i帧
			if (Pes(NextPack, leftlength, &NextPack, &leftlength, &PayloadData, &PayloadDataLen))
			{
				if (PayloadDataLen)
				{
					memcpy(buffer, PayloadData, PayloadDataLen);
					buffer += PayloadDataLen;
					*h264length += PayloadDataLen;
				}
			}
			else
			{
				if (PayloadDataLen)
				{
					memcpy(buffer, PayloadData, PayloadDataLen);
					buffer += PayloadDataLen;
					*h264length += PayloadDataLen;
				}

				break;
			}
		}
		else if (NextPack
			&& NextPack[0] == '\x00'
			&& NextPack[1] == '\x00'
			&& NextPack[2] == '\x01'
			&& NextPack[3] == '\xBC')
		{
			if (ProgramStreamMap(NextPack, leftlength, &NextPack, &leftlength, &PayloadData, &PayloadDataLen) == 0)
				continue;
		}
		else
		{
			//printf("no konw %x %x %x %x\n", NextPack[0], NextPack[1], NextPack[2], NextPack[3]);
			break;
		}
	}


	return *h264length;
}

Finally, attach the code download address related to this article https://download.csdn.net/download/wdglhack/15930839