IBM Cloud CDN - 流媒体加速(拉流场景)实战

目前IBM Cloud CDN 直播功能仅支持拉流场景，包括HLS和DASH两种协议。
关于CDN入门和具体功能可以参考在线文档，
https://cloud.ibm.com/docs/CDN?topic=CDN-getting-started

测试准备：
如果用户自带源站，只需准备以下origin服务器信息，可直接在IBM Cloud测试账户直接进行CDN配置，包括:

源站IP地址或者 Hostname
确认源站域名能被公网解析
访问URL具体路径，默认为站点根目录，i.e.: http://myorigin.danwcloud.com/hls
站点是否 HTTP SSL(https)加密，检查站点证书 -
https://www.sslshopper.com/ssl-checker.html

环境搭建：
这里会部署三个IBM Cloud 云主机和一个CDN服务，来实现一个直播流媒体拉流场景的端到端环境。为了测试方便，我们将使用nginx(web)和nginx-rtmp-module在一台东京的虚拟机上（8c16g）搭建HLS流媒体服务器，OBS模拟主播端从韩国做RTMP直推（IBM Cloud CDN目前无法支持推流），香港虚拟机作为客户端连接CDN的拉流边缘，查看播放和下载效果。

域名示例：
A. web站点域名 -> demopull.danwcloud.com
B. origin源站域名和IP -> myorigin.danwcloud.com / x.x.x.x
C. nginx-rtmp流媒体服务 -> myorigin.danwcloud.com / x.x.x.x
在这里插入图片描述

基本步骤：

OBS直播推流配置 (windows VSI - 首尔)
流媒体和源站搭建 (CentOS VSI - 东京)
IBM Cloud CDN服务创建和配置（IBM Cloud Portal）
客户端拉流设置（Ubuntu VSI - 香港）

1. OBS直播推流设置
OBS主播端到流媒体服务器将采用RTMP协议进行直推，不经过CDN。但一般生产环境下直播内容在传送到指定推流区域之前，还是会经过CDN边缘节点并进行相应转码处理，目前IBM Cloud尚无法支持这段路径

OBS下载和安装 - https://obsproject.com/download
Settings > Stream(指定RTMP远端服务)
Server: rtmp://nginx-rtmp-server-IP:1935/live
Stream Key: “stream”
在这里插入图片描述 Settings > Output: 选择高级参数，配置输出流

2. 流媒体服务和源站搭建
FFmpeg
FFmpeg可以用来记录、转换数字音频、视频，并能将其转化为流的开源程序(软编码),它包括了目前领先的音/视频编码库libavcodec。

[root@myorigin ~]# yum install epel-release
[root@myorigin ~]# yum localinstall --nogpgcheck https://download1.rpmfusion.org/free/el/rpmfusion-free-release-7.noarch.rpm
[root@myorigin ~]# yum install ffmpeg ffmpeg-devel
[root@myorigin ~]# ffmpeg -version
ffmpeg version 3.4.7 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-39)

nginx-rtmp
因为需要LuaJIT支持动态标头和 CDN 预加载功能，所以nginx的安装我们选择OpenResty版本而不是编译安装.

It is highly recommended to use OpenResty releases which bundle Nginx, ngx_lua (this module), LuaJIT, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with Nginx yourself since it is tricky to set up exactly right.Note that Nginx, LuaJIT, and OpenSSL official releases have various limitations and long standing bugs that can cause some of this module’s features to be disabled, not work properly, or run slower. Official OpenResty releases are recommended because they bundle OpenResty’s optimized LuaJIT 2.1 fork and Nginx/OpenSSL patches.

#OpenResty
[root@myorigin ~]# mkdir /root/workspace;cd /root/workspace
[root@myorigin workspace]# wget https://openresty.org/download/openresty-1.15.8.3.tar.gz
[root@myorigin workspace]# tar -zxvf openresty-1.15.8.3.tar.gz

#nginx-rtmp-module
[root@myorigin workspace]# git clone https://github.com/arut/nginx-rtmp-module.git
[root@myorigin workspace]# git clone https://github.com/arut/nginx-let-module.git

#install OpenResty with nginx-rtmp-module
[root@myorigin ~]# yum install -y gcc pcre pcre-devel openssl openssl-devel

[root@myorigin workspace]# cd /root/workspace/openresty-1.15.8.3

[root@myorigin openresty-1.15.8.3]# ./configure --add-module=../nginx-rtmp-module --add-module=../nginx-let-module -j2

[root@myorigin openresty-1.15.8.3]# make && make install

OpenResty nginx目录结构和配置文件

[root@myorigin ~]# ln -s /usr/local/openresty/nginx/conf/nginx.conf /root/nginx.conf
#nginx执行程序
[root@myorigin ~]# ln -s /usr/local/openresty/nginx/sbin/nginx /root/nginx
#nginx日志目录
[root@myorigin ~]# ln -s  /usr/local/openresty/nginx/logs/ /root/logs
#hls目录
[root@myorigin ~]# ln -s /usr/local/openresty/nginx/html/hls/ /root/hls
#dash目录
[root@myorigin ~]# ln  -s /usr/local/openresty/nginx/html/dash/ /root/dash

在这里插入图片描述
这里没办法附上nginx.conf的全部配置文件，只截取小段以供参考

a. RTMP转码、切片、分发逻辑

rtmp {
    server {
        listen 1935;
        chunk_size 4000;
        #Once receive stream, transcode for adaptive streaming
        #This single ffmpeg command takes the input and transforms
        #the source into 4 different streams with different bitrate
        #and quality. P.S. The scaling done here respects the aspect
        #ratio of the input.
        application live {
            live on; #Allows live input
            exec ffmpeg -i rtmp://localhost/$app/$name -async 1 -vsync -1
                        -c:v libx264 -c:a aac -b:v 256k -b:a 32k -vf "scale=480:trunc(ow/a/2)*2" -tune zerolatency -preset veryfast -crf 23 -f flv rtmp://localhost/hls/$name_480_low
                        -c:v libx264 -c:a aac -b:v 768k -b:a 96k -vf "scale=720:trunc(ow/a/2)*2" -tune zerolatency -preset veryfast -crf 23 -f flv rtmp://localhost/hls/$name_720_mid
                        -c:v libx264 -c:a aac -b:v 1024k -b:a 128k -vf "scale=960:trunc(ow/a/2)*2" -tune zerolatency -preset veryfast -crf 23 -f flv rtmp://localhost/hls/$name_960_high
                        -c:v libx264 -c:a aac -b:v 1920k -b:a 128k -vf "scale=1280:trunc(ow/a/2)*2" -tune zerolatency -preset veryfast -crf 23 -f flv rtmp://localhost/hls/$name_1280_HD
                        -c copy -f flv rtmp://localhost/hls/$name_src;
         
        #Split the stream into HLS fragments
        application hls {
            live on; # Allows live input from above
            hls on; # Enable HTTP Live Streaming

            #Pointing this to an SSD is better as this involves lots of IO
            hls_path /usr/local/openresty/nginx/html/hls;
            hls_fragment 3;
            hls_playlist_length 15;

            #Instruct clients to adjust resolution according to bandwidth
            hls_variant _480_low BANDWIDTH=288000 RESOLUTION=480x270; # Low bitrate, sub-SD resolution
            hls_variant _720_mid BANDWIDTH=448000 RESOLUTION=720x480; # Medium bitrate, SD resolution
            hls_variant _960_high BANDWIDTH=1152000 RESOLUTION=960x540; # High bitrate, higher-than-SD resolution
            hls_variant _1280_HD BANDWIDTH=2048000 RESOLUTION=1280x720; # High bitrate, HD 720p resolution
            hls_variant _src BANDWIDTH=4096000; # Source bitrate, source resolution
        }
    }
}

b. header添加以及CDN预加热支持

      location ~ ^/hls/(.+)-(.+)\.ts$ {
            types {
               application/x-mpegURL m3u8;
               video/mp2t ts;
            }
            #let $nodeIndex $2 + 1;
            #let $nextNode $1 . $nodeIndex . ".ts";
            lua_code_cache off;
            set_by_lua_file $cdn_proloading_header /usr/local/openresty/nginx/lua_script/customer_header.lua  $1;
            add_header CDN-Origin-Assist-Prefetch-Path $cdn_proloading_header;

            root /usr/local/openresty/nginx/html;
            #add_header CDN-Origin-Assist-Prefetch-Path $nextNode;
            add_header Access-Control-Allow-Origin *;
            add_header Access-Control-Allow-Methods 'GET, POST, OPTIONS';
       }

       location ~ ^/hls/(.+)\.m3u8 {
           types {
               application/x-mpegURL m3u8;
           }
           lua_code_cache off;
           set_by_lua_file $cdn_proloading_header /usr/local/openresty/nginx/lua_script/customer_header.lua  $1;
           add_header CDN-Origin-Assist-Prefetch-Path $cdn_proloading_header;
           add_header Access-Control-Allow-Origin *;
           add_header Access-Control-Allow-Methods 'GET, POST, OPTIONS';
           root /usr/local/openresty/nginx/html;
       }

       location = /hls/stream.m3u8 {
           types {
               application/x-mpegURL m3u8;
           }
           add_header Access-Control-Allow-Origin *;
           add_header Access-Control-Allow-Methods 'GET, POST, OPTIONS';

           root /usr/local/openresty/nginx/html;
       }

certbot生成web站点证书
请参考 https://certbot.eff.org/lets-encrypt/centosrhel7-nginx

在测试环境中我们将用OpenResty版本的nginx作为分发源站的web服务，这里安装的标准nginx只是为了让certbot判断nginx状态，用于证书生成，千万不要覆盖自定义的OpenResty nginx.conf.

[root@myorigin ~]# yum install certbot python2-certbot-nginx nginx
[root@myorigin ~]# certbot certonly --nginx
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: myorigin.danwcloud.com
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel):

按照提示创建好证书，然后检查 /etc/letsencrypt/renewal/myorigin.danwcloud.com.conf

[root@myorigin ~]# cat /etc/letsencrypt/renewal/myorigin.danwcloud.com.conf
#renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/myorigin.danwcloud.com
cert = /etc/letsencrypt/live/myorigin.danwcloud.com/cert.pem
privkey = /etc/letsencrypt/live/myorigin.danwcloud.com/privkey.pem
chain = /etc/letsencrypt/live/myorigin.danwcloud.com/chain.pem
fullchain = /etc/letsencrypt/live/myorigin.danwcloud.com/fullchain.pem

#Options used in the renewal process
[renewalparams]
account = 5baee9da17b293942abc1ae88a442722
authenticator = nginx
installer = nginx
server = https://acme-v02.api.letsencrypt.org/directory

编辑nginx.conf 添加证书（如果http测试，此步省略）
在这里插入图片描述
最后通过sslchecker 检查站点证书是否工作正常

测试nginx配置并启动服务

 #test nginx conf(OpenResty)
[root@myorigin ~]# /root/nginx -t
nginx: the configuration file /usr/local/openresty/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/openresty/nginx/conf/nginx.conf test is successful

#start Nginx
[root@myorigin ~]# /root/nginx
#stop
[root@myorigin ~]# /root/nginx -s stop
#restart
[root@myorigin ~]# /root/nginx -s reload
#check listening ports and process
[root@myorigin ~]# netstat -anp |grep -E "1935|443”; ps aux |grep nginx

测试辅助脚本 – 后台运行/root/syn.sh, 获取最新视频片段文件名放在临时文件，通过lua脚本“customer_header”让nginx响应m3u8 http请求时添加到header中，给CDN预加载使用.

#获取片段文件名
[root@myorigin ~]# cat syn.sh
#!/bin/bash
content_dir="/usr/local/openresty/nginx/html/hls"
while true
do
for m3u8 in `ls $content_dir |grep m3u8 |grep _ `
do
last_segment=`tail -n 1 ${content_dir}/${m3u8}`
a=`perl -pe 's/(\d+)(?!.*\d+)/$1/e' <<< $last_segment`
echo  -n $a > /tmp/${m3u8}
done
sleep 1
done

[root@myorigin ~]# cd /root; nohup ./syn.sh &

#添加HLS header
[root@myorigin ~]# cat /usr/local/openresty/nginx/lua_script/customer_header.lua
local open = io.open
local function read_file(path)
    local file = open(path, "rb") -- r read mode and b binary mode
    if not file then return nil end
    local content = file:read "*a" -- *a or *all reads the whole file
    file:close()
    return content
end
return read_file("/tmp/" .. ngx.arg[1] .. ".m3u8");
--return ngx.arg[1] .. s;

模拟本地推流，执行curl测试

[root@myorigin ~]# cat /root/record.sh
ffmpeg -re -stream_loop -1 -i /root/africa.mkv -vcodec libx264 -acodec aac -strict -2 -f flv  rtmp://localhost/live/stream
[root@myorigin ~]# nohup /root/record.sh &
[root@myorigin ~]# watch -n 1 cat /tmp/stream_720_mid.m3u8

#curl to check if nginx web works 
danws-MacBook-Pro:~ danw$ curl https://myorigin.danwcloud.com/hls/stream.m3u8

#stop local test streaming
[root@2lnpag9a hls]# pkill ffmpeg

到这，源站流媒体和web服务基本完成，接下来我们看看CDN的配置

3. IBM Cloud CDN配置
IBM云控制台很简洁，在商品“目录”里搜索“content delivery“关键字，点击“创建”即可
在这里插入图片描述
输入CDN要加速的域名和源站信息

这里配置用到的基本概念和业界通用，这里我将采用DV SAN证书：

Hostname: 流媒体web前端域名（此域名用来指向非origin的web站点，作为CDN服务的主标识）
自定义cname:用于web前端的cname域名（如未提供,系统自动以cdn.appdomain.cloud为尾缀添加cname；此cname记录需要在服务web站点的DNS服务器上添加）
origin配置: 可留空缺省（服务创建完成后，可根据站点目录或路径信息自行添加）
Server（源站）: 输入源站的域名或者IP地址 (如有点播存放在IBM云对象存储，可直接选择Object Storage)
HTTP 和HTTPS端口 : 可自定义CDN访问源站的端口（此处为默认80和443）
SSL证书类型（此测试中我们选择DV SAN证书）

通配符证书：所有客户都使用在供应商 CDN 网络上部署的同一证书，必须使用 CNAME（包括 IBM 后缀 .cdn.appdomain.cloud）来访问服务。例如，https://www.example-cname.cdn.appdomain.cloud

DV SAN证书：对于 SAN 证书，多个客户域通过将其域名添加到 SAN 条目来共享单个 SAN 证书，可以使用主机名来访问服务，例如 https://www.example.com。CDN服务从创建到就绪会持续几个小时，因为DV SAN完成 HTTPS多域控制验证需经历以下六个状态，其中证书分发部署到边缘服务器(Deploying cert)需要2到4小时

当CDN服务状态变为running（运行），则代表它已就绪
在这里插入图片描述
添加源站目录路径
直播HLS拉流选择VOD，开工单联系IBM后台，后台会将站点目录的live streaming功能激活（10分钟左右）

CDN拉流定价
Live Streaming功能与“Static”计费方式一致，按CDN各区域的边缘节点处理数据带宽总量计算）
https://cloud.ibm.com/docs/CDN?topic=CDN-pricing#cdn-static-bandwidth

4.客户端拉流设置
在HK虚拟机上创建一个本地html网页（需要 javascript hls支持）

<!DOCTYPE html>
<html>
<head>
<meta charset=utf-8 />
<title>HLS-video</title>
<link href="https://unpkg.com/video.js/dist/video-js.css" rel="stylesheet">
</head>
<body>
  <h1>HLS Video Play</h1>
  <video-js id="origin_video" class="vjs-default-skin" controls preload="auto" width="640" height="268">
    <source src="https://myorigin.danwcloud.com/hls/stream_720_mid.m3u8" type="application/x-mpegURL">
  </video-js>
  <video-js id="cdn_video" class="vjs-default-skin" controls preload="auto" width="640" height="268">
    <source src="https://demopull.danwcloud.com/hls/stream_720_mid.m3u8" type="application/x-mpegURL">
  </video-js>
  <script src="https://unpkg.com/video.js/dist/video.js"></script>
  <script src="https://unpkg.com/@videojs/http-streaming/dist/videojs-http-streaming.js"></script>
  <script>
    var player = videojs('origin_video');
       var player = videojs('cdn_video ');
  </script>
</body>
</html>

最后在首尔视频采集端启动OBS推流，在客户端浏览器打开网页，即可直观地体验从源站和CDN请求视频播放的效果，这里提供的是一套论证小实验（这个时候CDN未必能有源站快，如有很多客户端请求，后面的就享福了），当然如果用户想降低CDN回源率以及跨区域的播放质量，那么源站到CDN的预热机制需要优化，再就是源站区域节点覆盖一定要到位。
在这里插入图片描述
happy learning !!!

IBM Cloud CDN - 流媒体加速(拉流场景)实战

猜你喜欢