FFmpeg+SDL---SDL video display

Chapter Four SDL Video Display

It is recommended to read before reading this chapter: FFmpeg+SDL-----Syllabus

table of Contents

• Video Display knowledge
• SDL Introduction
• VC set up under SDL development environment
• sample program run
• Function SDL video display
• SDL video display data structures
• Advanced - a sample program run
• Advanced -SDL in multithreading and events
• Exercise

Video display knowledge

Video display process : The video display process is the process of "drawing" pixel data on the screen. For example, to display YUV is to "draw" YUV in the window of the system.

Introduction to SDL

• effect
  • The function of the SDL (Simple DirectMedia Layer) library is to encapsulate the complex video and audio bottom interactive work and simplify the difficulty of video and audio processing.
  • This course only involves a small part of the SDL library-the video display part. (This library is mainly used for game development)
• Features
  • Cross-platform, the program can be used on multiple platforms
  • Open source
• structure
  • The SDL structure is shown below. It can be seen that it actually calls the underlying APIs such as DirectX to complete the interaction with the hardware. Call the corresponding API according to different systems.
    

Construction of SDL development environment under VC

• New console project
  • Open VC++
  • File->New->Project->Win32 Console Application
• Copy SDL development files
  • Copy the header file (*.h) to the include subfolder of the project folder
  • Import the library file (*.lib) and copy it to the lib subfolder of the project folder
  • Copy the dynamic library file (*.dll) to the project folder
• Configure development files
  • Open the properties panel
    • Solution Explorer -> Right click on the project -> Properties
• Header file configuration
  • Configuration Properties ->C/C+±>General ->Additional include directory, enter "include" (the directory where the file was copied just now)
• Import library configuration
  • Configure Properties -> Linker -> General -> Additional Library Directory, enter "lib" (the directory where the file was copied just now)
  • Configuration properties -> linker -> input -> additional dependencies, input "SDL2.lib; SDL2main.lib" (the file name of the imported library)
• Dynamic library does not need to be configured

test

• Create source code file
  • Create a C/C++ file containing the main() function in the project (if you already have one, you can skip this step), and write the source code in this file in the subsequent steps.
• Include header file
  • If SDL is used in C language, use the following code
    #include “SDL2/SDL.h”
  • If SDL is used in C++ language, use the following code
    extern “C”
    { #include “SDL2/SDL.h” }
    
    
• A SDL interface function is called in main(). For example, the following code initializes SDL

int main(int argc, char* argv[]){
	if(SDL_Init(SDL_INIT_VIDEO)) { 
		printf( "Could not initialize SDL - %s\n", SDL_GetError()); 
	} else{
		printf("Success init SDL");
	}
	return 0;
}

If the operation is correct, it means that SDL has been configured

Sample program run

SDL video display function

1. The flow chart of SDL video display is shown below

2. Introduction to SDL video display function
▫ SDL_Init(): initialize the SDL system
▫ SDL_CreateWindow(): create a window SD L_Window
▫ SDL_CreateRenderer(): create a renderer SDL_Renderer
▫ SDL_CreateTexture(): create Texture SDL_Texture
▫ SDL_UpdateTexture(): set the texture
data▫ SDL_RenderCopy(): copy the texture data to the renderer▫
SDL_RenderPresent():
display▫ SDL_Delay(): utility function, used for delay.
▫ SDL_Quit(): Quit the SDL system

SDL video display data structure

1. The data structure of SDL video display is as follows

2. Introduction to SDL data structure
▫ SDL_Window: represents a "window", a window can play multiple videos at the same time
▫ SDL_Renderer: represents a "renderer"
▫ SDL_Texture: represents A "texture" and a texture actually correspond to a YUV
▫ SDL_Rect: A simple rectangular structure, confirm where it is drawn, it can be in the form of 2*2, not just one

Source code 1:

#include <stdio.h>

extern "C"
{
#include "sdl/SDL.h"
};

const int bpp=12;

int screen_w=640,screen_h=360;		//修改值可以调节屏幕的大小
const int pixel_w=640,pixel_h=360;

unsigned char buffer[pixel_w*pixel_h*bpp/8];

int main(int argc, char* argv[])
{
	if(SDL_Init(SDL_INIT_VIDEO)) {  
		printf( "Could not initialize SDL - %s\n", SDL_GetError()); 
		return -1;
	} 

	SDL_Window *screen; 
	//SDL 2.0 Support for multiple windows
	screen = SDL_CreateWindow("Simplest Video Play SDL2", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED,
		screen_w, screen_h,SDL_WINDOW_OPENGL|SDL_WINDOW_RESIZABLE);
	if(!screen) {  
		printf("SDL: could not create window - exiting:%s\n",SDL_GetError());  
		return -1;
	}
	SDL_Renderer* sdlRenderer = SDL_CreateRenderer(screen, -1, 0);  

	Uint32 pixformat=0;
	//IYUV: Y + U + V  (3 planes)
	//YV12: Y + V + U  (3 planes)
	pixformat= SDL_PIXELFORMAT_IYUV;  

	SDL_Texture* sdlTexture = SDL_CreateTexture(sdlRenderer,pixformat, SDL_TEXTUREACCESS_STREAMING,pixel_w,pixel_h);

	FILE *fp=NULL;
	fp=fopen("sintel_640_360.yuv","rb+");

	if(fp==NULL){
		printf("cannot open this file\n");
		return -1;
	}

	SDL_Rect sdlRect;  

	while(1){
			if (fread(buffer, 1, pixel_w*pixel_h*bpp/8, fp) != pixel_w*pixel_h*bpp/8){
				// Loop
				fseek(fp, 0, SEEK_SET);
				fread(buffer, 1, pixel_w*pixel_h*bpp/8, fp);
			}

			SDL_UpdateTexture( sdlTexture, NULL, buffer, pixel_w);  

			/* 修改下面的值可以调节视频显示的坐标，修改为多分割显示不同的码流 */
			sdlRect.x = 0;  
			sdlRect.y = 0;  
			sdlRect.w = screen_w;  		
			sdlRect.h = screen_h;  
			
			SDL_RenderClear( sdlRenderer );   
			SDL_RenderCopy( sdlRenderer, sdlTexture, NULL, &sdlRect);  
			SDL_RenderPresent( sdlRenderer );  
			//Delay 40ms
			SDL_Delay(40);
			
	}
	SDL_Quit();
	return 0;
}

Advanced-events and multithreading in SDL

1. SDL multithreading▫
function
 SDL_CreateThread(): create a thread
▫ data structure
 SDL_Thread: thread handle
2. SDL event
▫ function
 SDL_WaitEvent() wait for an event
 SDL_PushEvent() send an event
▫ data structure
 SDL_Event: Represents an event

Advanced source code

A new thread is added to manually stretch the width of the screen, and the size of the video will also be dynamically adjusted accordingly.

#include <stdio.h>

extern "C"
{
#include "sdl/SDL.h"
};

const int bpp=12;

int screen_w=500,screen_h=500;
const int pixel_w=320,pixel_h=180;

unsigned char buffer[pixel_w*pixel_h*bpp/8];

//Refresh Event
#define REFRESH_EVENT  (SDL_USEREVENT + 1)
//Break
#define BREAK_EVENT  (SDL_USEREVENT + 2)

int thread_exit=0;

int refresh_video(void *opaque){
	thread_exit=0;
	while (thread_exit==0) {
		SDL_Event event;
		event.type = REFRESH_EVENT;
		SDL_PushEvent(&event);
		SDL_Delay(40);
	}
	thread_exit=0;
	//Break
	SDL_Event event;
	event.type = BREAK_EVENT;
	SDL_PushEvent(&event);
	return 0;
}

int main(int argc, char* argv[])
{
	if(SDL_Init(SDL_INIT_VIDEO)) {  
		printf( "Could not initialize SDL - %s\n", SDL_GetError()); 
		return -1;
	} 

	SDL_Window *screen; 
	//SDL 2.0 Support for multiple windows
	screen = SDL_CreateWindow("Simplest Video Play SDL2", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED,
		screen_w, screen_h,SDL_WINDOW_OPENGL|SDL_WINDOW_RESIZABLE);
	if(!screen) {  
		printf("SDL: could not create window - exiting:%s\n",SDL_GetError());  
		return -1;
	}
	SDL_Renderer* sdlRenderer = SDL_CreateRenderer(screen, -1, 0);  

	Uint32 pixformat=0;
	//IYUV: Y + U + V  (3 planes)
	//YV12: Y + V + U  (3 planes)
	pixformat= SDL_PIXELFORMAT_IYUV;  

	SDL_Texture* sdlTexture = SDL_CreateTexture(sdlRenderer,pixformat, SDL_TEXTUREACCESS_STREAMING,pixel_w,pixel_h);

	FILE *fp=NULL;
	fp=fopen("test_yuv420p_320x180.yuv","rb+");

	if(fp==NULL){
		printf("cannot open this file\n");
		return -1;
	}

	SDL_Rect sdlRect;  

	SDL_Thread *refresh_thread = SDL_CreateThread(refresh_video,NULL,NULL);
	SDL_Event event;
	while(1){
		//Wait
		SDL_WaitEvent(&event);
		if(event.type==REFRESH_EVENT){
			if (fread(buffer, 1, pixel_w*pixel_h*bpp/8, fp) != pixel_w*pixel_h*bpp/8){
				// Loop
				fseek(fp, 0, SEEK_SET);
				fread(buffer, 1, pixel_w*pixel_h*bpp/8, fp);
			}

			SDL_UpdateTexture( sdlTexture, NULL, buffer, pixel_w);  

			//FIX: If window is resize
			sdlRect.x = 0;  
			sdlRect.y = 0;  
			sdlRect.w = screen_w;  
			sdlRect.h = screen_h;  
			
			SDL_RenderClear( sdlRenderer );   
			SDL_RenderCopy( sdlRenderer, sdlTexture, NULL, &sdlRect);  
			SDL_RenderPresent( sdlRenderer );  
			
		}else if(event.type==SDL_WINDOWEVENT){
			//If Resize
			SDL_GetWindowSize(screen,&screen_w,&screen_h);
		}else if(event.type==SDL_QUIT){
			thread_exit=1;			//修改全局变量，下一次循环接收到的事件是BREAK_EVENT，然后退出整个循环过程
		}else if(event.type==BREAK_EVENT){
			break;
		}
	}
	SDL_Quit();
	return 0;
}