项目记录：MPEG-DASH整理3
该项目的目标是 基于3DOF的视频内容 实现一个6DOF的播放器.
观看者戴着VR头显,在位置发生运动之后,播放器能够根据其位置的不同,下载不同位置的码流,进行解码渲染.简单地实现6DOF.
服务器端:
-
服务器端必须能够有多个视点位置的全景视频,作为不同的
AdaptationSet存储起来. -
那么客户端如何知道服务器端的各个
AdaptationSet的位置呢?我通过给每个
AdaptationSet增加一个位置信息的字段.- 比如
<Viewpoint schemeIdUri="urn:mpeg:mpegI:omaf:2018:vwpt" value="0,0,0,0"/>
- 比如
-
以下是生成的
MPD表示例:<?xml version="1.0" encoding="utf-8"?> <MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xlink="http://www.w3.org/1999/xlink" xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd" profiles="urn:mpeg:dash:profile:isoff-live:2011" type="static" mediaPresentationDuration="PT15.0S" minBufferTime="PT0.0S" maxSegmentDuration="PT1.0S"> <ProgramInformation moreInformationURL="http://127.0.0.1/"> <Title>seg\1.mpd generated by GPAC</Title> </ProgramInformation> <BaseURL>http://127.0.0.1/</BaseURL> <Period duration="PT15.0S"> <AdaptationSet segmentAlignment="true" bitstreamSwitching="true" maxWidth="3840" maxHeight="1920" maxFrameRate="30/1" par="16:9" lang="und"> <Viewpoint schemeIdUri="urn:mpeg:mpegI:omaf:2018:vwpt" value="0,0,0,0"/> <Representation id="1" mimeType="video/mp4" codecs="hev1.010101" width="3840" height="1920" frameRate="30" sar="1:1" startWithSAP="1" bandwidth="1504040"> <SegmentList timescale="1200000" duration="48000"> <Initialization sourceURL="TestStream0_ViewPoint0/init.mp4" /> <SegmentURL media="TestStream0_ViewPoint0/1.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/2.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/3.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/4.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/5.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/6.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/7.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/8.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/9.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/10.m4s" /> </SegmentList> </Representation> </AdaptationSet> <AdaptationSet segmentAlignment="true" bitstreamSwitching="true" maxWidth="3840" maxHeight="1920" maxFrameRate="30/1" par="16:9" lang="und"> <Viewpoint schemeIdUri="urn:mpeg:mpegI:omaf:2018:vwpt" value="1,1,1,1"/> <Representation id="1" mimeType="video/mp4" codecs="hev1.010101" width="3840" height="1920" frameRate="30" sar="1:1" startWithSAP="1" bandwidth="7508880"> <SegmentList timescale="1200000" duration="48000"> <Initialization sourceURL="TestStream1_ViewPoint1/init.mp4" /> <SegmentURL media="TestStream1_ViewPoint1/1.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/2.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/3.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/4.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/5.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/6.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/7.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/8.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/9.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/10.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/11.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/12.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/13.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/14.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/15.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/16.m4s" /> </SegmentList> </Representation> </AdaptationSet> </Period> </MPD>
客户端:
- 客户端通过解析
MPD表能够获得得到所有AdaptationSet的ViewPoint,进行存储 - 在检测到用户的位置发生变化时,进行与这些参考位置进行比较. 选择适合的视频码流进行下载.
ViewPoint 字段解析
-
IAdaptationSet类中提供了很多IDescriptor的接口virtual const std::vector<IDescriptor *>& GetViewpoint () const = 0;通过
GetViewpoint这个接口获得MPD中给每个AdaptationSet新增的 视点位置 -
于是,新建一个类来管理这些
ViewPoint,一个ViewPoint表示当前VR头盔对应的位置.
dash::mpd::ViewPointManager *viewpointManager; // 管理MPD表所有的ViewPoint
dash::mpd::ViewPoint *currentViewPoint; // 表示当前的ViewPoint
IViewPointManager.h
/**
* @class dash::mpd::IViewPointManager
* @brief the Class to Manage the ViewPoint of every AdaptationSet
* @author JZChen
* Email: [email protected]
* @version 1.0
* @date 2018.11.29
* @copyright Xidian MMC 203, All Rights Reserved \n\n
* This source code and its use and distribution, is subject to the terms
* and conditions of the applicable license agreement.
*/
#ifndef VIEWPOINTMANGER_H_
#define VIEWPOINTMANGER_H_
#include "config.h"
#include "IAdaptationSet.h"
namespace dash {
namespace mpd {
class ViewPoint {
public:
ViewPoint() :viewId(0), centerX(0), centerY(0), centerZ(0) {}
ViewPoint(int id,int x,int y,int z) :viewId(id), centerX(x), centerY(y), centerZ(z) {}
ViewPoint(const ViewPoint& rh) :viewId(rh.viewId), centerX(rh.centerX), centerY(rh.centerY), centerZ(centerZ) {}
void operator=(const ViewPoint& rh) {
this->viewId = rh.viewId;
this->centerX = rh.centerX;
this->centerY = rh.centerY;
this->centerZ = rh.centerZ;
}
void SetViewPoint(int id, int x, int y, int z) {
this->viewId = id;
this->centerX = x;
this->centerY = y;
this->centerZ = z;
}
ViewPoint(IDescriptor *pDes) {
ViewPoint();
std::string tmp = pDes->GetValue();
printf("%s\n", tmp);
// ViewPoint的Value格式如下: ViewID,CenterX,CenterY,CenterZ
std::string::iterator it_begin;
std::string::iterator it_end;
it_begin = tmp.begin();
it_end = find(tmp.begin(), tmp.end(), ',');
viewId = std::stoi(tmp.substr(it_begin - tmp.begin(), it_end - it_begin));
it_begin = ++it_end;
it_end = find(tmp.begin(), tmp.end(), ',');
centerX = std::stoi(tmp.substr(it_begin - tmp.begin(), it_end - it_begin));
it_begin = ++it_end;
it_end = find(tmp.begin(), tmp.end(), ',');
centerY = std::stoi(tmp.substr(it_begin - tmp.begin(), it_end - it_begin));
it_begin = ++it_end;
it_end = find(tmp.begin(), tmp.end(), ',');
centerZ = std::stoi(tmp.substr(it_begin - tmp.begin(), it_end - it_begin));
}
// TODO:精细化这个判断条件
bool operator==(const ViewPoint& rh) {
if (centerX == rh.centerX&&
centerY == rh.centerY&&
centerZ == rh.centerZ) {
return true;
}
else {
return false;
}
}
int viewId;
int centerX;
int centerY;
int centerZ;
};
}
}
namespace dash {
namespace mpd {
class ViewPointManager {
public:
ViewPointManager():SizeOfAdaptationSet(0), SizeOfViewPoint(0){
}
ViewPointManager(std::vector<IAdaptationSet*> AdaptationSetList) {
ViewPointManager();
SizeOfAdaptationSet = AdaptationSetList.size();
this->AdaptationSetList = AdaptationSetList;
}
std::vector<IAdaptationSet*> AdaptationSetList;
std::vector<ViewPoint*> ViewPointList; // 与AdaptationSetList一一对应
int SizeOfAdaptationSet; // 一共有多少个AdaptationSet
int SizeOfViewPoint; // 一个多少个ViewPoint,
// ================ Get Functions ====================== //
const std::vector<IAdaptationSet*>& GetAdaptationSetList() const {
return AdaptationSetList;
}
const std::vector<ViewPoint*>& GetViewPointList() const {
return ViewPointList;
}
// ================ Set Functions ====================== //
bool SetViewPointList(const std::vector<IAdaptationSet*> &AdaptationSetList) {
this->AdaptationSetList = AdaptationSetList;
return SetViewPointList();
}
bool SetViewPointList() {
// <1> 获得到所有的GetAdaptationSet
SizeOfAdaptationSet = AdaptationSetList.size();
for (int i = 0; i < SizeOfAdaptationSet; ++i) {
/// 一般一个AdaptationSet只能有一个位置
auto DesViewpointList = AdaptationSetList[i]->GetViewpoint();
// <2> 获得所有的DescriptionDesViewpointList
auto SizeOfDesViewpointList = DesViewpointList.size();
for (int j = 0; j < SizeOfDesViewpointList; ++j) {
auto DesViewpoint = DesViewpointList[j];
// <3> 存储所有的ViewPoint
ViewPointList.push_back(new ViewPoint(DesViewpoint));
}
}
if (ViewPointList.size())
SizeOfViewPoint = ViewPointList.size();
if (SizeOfViewPoint <= 0) {
std::cout << "Exception at SetViewPointList" << std::endl;
return false;
}
return true;
}
/// tobeContinued
};
}
}
#endif
- 通过这个类我们就可以管理所有的VIewPoint了.在位置发生变化的时候,取到对应的ViewPoint的Adaptation.
- 现在还没有写好.应该改成
Map存储.
切换下载视频源
-
在
DoBuffering线程中,需要根据当前头盔的位置,进行选择判断.// 根据number,获得新的媒体段 media = dashreceiver->logic->GetSegment(number); -
可以通过遍历
ViewPointManager中的视点位置与当前的视点位置进行比较进行下载更新Segment