Qt to write pictures and video TCP / UDP network transmission

I. Introduction

Many years ago, has done similar projects, nothing more than the local picture uploaded to the server, so simple, in fact, with post http upload relatively simple and easy, without having to custom protocols, binary data can be directly set up, while the use of TCP or UDP traffic, it must custom protocols, because they do not know when the data is received over a complete picture of the data, possibly at the same time sending a lot of picture data, but can not distinguish between images received which client sent, TCP long connection would also need to detect a heartbeat connection, it must be customized to support a communication protocol, this protocol uses a communication protocol format Shanghai supervision platform, relatively strong expansion, wherein the head information comprises a current type + the length of the complete data packet, this is the type of communication protocol identifier, so that next to other types such as a building intercom call can IDOOR, the server can know what the basis of this identifier parsing algorithm to process the subsequent data, and full length of the current data packet may be used to process the received data, only It represents a data length of one complete picture reception data, decoding processing again. When the transmission of images to a certain speed, such as when a second transmission 20 pictures, in fact, is equivalent to the transmission of video, and generally the human eye to see a 20 second video is a picture basically know.

TCP theoretically stable connection, without loss, not just the middle of a packet into a packet, certainly ensure the integrity of a data packet, at the two TCP connections, one is connected to long, once has been connected to the communication, mainly used in scenes such as the frequent communication in real-time upload, there is still a short connection, the client or the server finished data reception complete data on disconnected immediately, mainly used in infrequent communication scenarios in such upload alarm, the alarm in the case rarely occurs after day, preferably using a short connection, the system can save a lot of overhead, the Qt TCP communication of the package is also useful, in just a few small concurrency dozens of connected projects, efficiency is still possible, is said to QNetwork components of the underlying Qt5 rewrite, and more efficient than some of Qt4, I am useless to the detailed view of the corresponding source code, only to be told. Qt network traffic class, we usually used is three: QTcpSocket client class, QTcpServer server class, QUdpSocket communications class, why not QUdpServer class? In fact, UDP is a communication connection, small footprint, he can be both a client may also be a server, specify if you want to call the bind method can be used as the server port. This program supports both the TCP and UDP mode mode, the actual test down, it is recommended to use TCP mode, UDP mode because no connection to send large amounts of data packets in a short time found to packet loss, and packet size is limited, is 65507 words Day, about 64K, so the real-time transmission of UDP mode picture resolution is not too large, measured 640 * 480 video files is still quite good, 720P basically a bit miserable, dropping a lot, you may also need to improve the post-processing from the agreement .

This procedure and the agreement images using base64 encoding, after receiving the decoded image generated base64 string, the QByteArray turned into built-in class toBase64 method base64 encoded string, QByteArray :: fromBase64 method base64 string is reduced to the data. After the experiment many times through statistical data show that the codec can also speed, which 720P picture coding 25ms-30ms, decoding 15ms-20ms, 1080P picture coding 35ms-40ms, decoding 25ms-30ms. On the whole, a second transmission and decoding pictures 25-30 25-30 pictures, still no problem, just take the CPU codec, if the number of channels to open more, then, is very CPU-intensive, but to cope with some simple scenarios like a duck or no pressure.

Experience Address: https://pan.baidu.com/s/1bbL2ZughZAgfIGrexyN-9g extraction code: zkeh File name: bin_video_image.zip.

Second, the functional characteristics

Multithreading send and receive pictures and data parsing picture data, the card is not the main interface.
It supports both TCP and UDP modes, encapsulates the TCP mode, and client and server class category UDP mode.
Picture Transfer clients simultaneously sent to multiple server support can be used as a teacher's computer screen to send the same scenario multiple student machines.
Support multiple clients simultaneously transmit images to the server, each server connection will automatically open thread send and receive pictures and parse data.
Custom label control signal slots mechanism to draw pictures, the card is not the main interface.
Analyzing heartbeat mechanism comes off, the server automatically reconnect can set the timeout.
Each message has a unique message identifier uuid, the server will return later received uuid corresponds to the message acknowledged receipt, the client can determine based on this return message server successfully resolved, without recurrence, ensuring that data sent out server received and successfully resolved.
Each message has a unique image identifier In Flag, corresponding to the ID number, this identity is determined according to which interface display needs to parse.
Image transmission base64 string format, the receiving end receives image regenerated after base64 decoding picture data string.
Send and receive all of the data signal has issued, easy to view output.
Offer singleton class, there is only one convenience when used without new.
Self xml protocol definition, are free to develop other properties, such as field to bring pictures and other content.

Third, the communication protocol

Long connection uses TCP and UDP protocols optional, default communication port 6000.
Xml using custom communication protocol.
All transport header plus 20 bytes: IIMAGE: 0000000000000, IIMAGE: a fixed head, followed by the contents of the 13 bytes length (including the length of the head 20) string.
Part protocol header bytes omitted below.
Data returned from the server in the corresponding message is received uuid the uuid.
When the server returned each time have brought the current time, the client can be used for the school.

客户端发送心跳
<?xml version="1.0" encoding="UTF-8"?>
<ImageClient Uuid="8AF12208-0356-434C-8A49-69A2170D9B5A" Flag="SHJC00000001">
    <ClientHeart/>
</ImageClient>

服务器收到心跳返回
<?xml version="1.0" encoding="UTF-8"?>
<ImageServer Uuid="8AF12208-0356-434C-8A49-69A2170D9B5A" NowTime="2019-12-05 16:37:47">
    Ok
</ImageServer>

客户端发送图片
<?xml version="1.0" encoding="UTF-8"?>
<ImageClient Uuid="66BCB44A-B567-48ED-8889-36B8FA6C4363" Flag="SHJC00000001">
    <ClientImage>图片base64编码后的字符串/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAJAAtADASIAAhEBAxEB/8QAHwAAAQUBAQEB...nvWsQRlXA61mTjmtWcdazLgcmrQ0U2plSMKjpDE7UtFFAwxRRRQAUuKWigQlFFFLcD//2Q==</ClientImage>
</ImageClient>

服务端收到图片返回
<?xml version="1.0" encoding="UTF-8"?>
<ImageServer Uuid="66BCB44A-B567-48ED-8889-36B8FA6C4363" NowTime="2019-12-05 16:38:47">
    Ack
</ImageServer>

Fourth, renderings

Fifth, the core code

//图片转base64字符串
QByteArray DeviceFun::getImageData2(const QImage &image)
{
    QByteArray imageData;
    QBuffer buffer(&imageData);
    image.save(&buffer, "jpg");
    imageData = imageData.toBase64();
    return imageData;
}

//base64字符串转图片
QImage DeviceFun::getImage(const QString &data)
{
    QByteArray imageData = QByteArray::fromBase64(data.toLatin1());
    QImage image;
    image.loadFromData(imageData);
    return image;
}

//客户端线程发送图片
void TcpImageClient::run()
{
    while(!stopped) {
        //这里采用线程去处理,其实完全可以用定时器搞定,毕竟tcp的write是异步的,操作系统自动调度
        //为了后期的拓展性,比如需要判断是否发送成功之类的,需要同步处理,所以改成的线程去处理
        //图片数据转成base64编码的数据也需要时间的,主要的耗时在转码
        //取出数据发送,这里需要加锁,避免正在插入数据
        if (isOk && images.count() > 0) {
            QMutexLocker locker(&mutexImage);
            QImage image = images.takeFirst();
            QString imageData = DeviceFun::getImageData(image);
            emit readyWrite(imageData);
        }

        //要稍微休息下,否则CPU会被一直占用
        msleep(1);
    }

    stopped = false;
}

//服务端线程解析图片
void TcpImageSocket::run()
{
    while(!stopped) {
        //这里采用线程去处理,其实完全可以用定时器搞定,毕竟tcp的write是异步的,操作系统自动调度
        //为了后期的拓展性,比如需要判断是否发送成功之类的,需要同步处理,所以改成的线程去处理
        //base64编码数据转图片数据也需要时间的,主要的耗时在转码
        //取出数据发送,这里需要加锁,避免正在插入数据
        if (imageFlags.count() > 0) {
            QMutexLocker locker(&mutexImage);
            QString imageFlag = imageFlags.takeFirst();
            QString imageData = imageDatas.takeFirst();
            QImage image = DeviceFun::getImage(imageData);
            emit receiveImage(imageFlag, image);
        }

        //要稍微休息下,否则CPU会被一直占用
        msleep(1);
    }

    stopped = false;
}

//客户端类使用方法
TcpImageClient *client = new TcpImageClient(this);
UdpImageClient *client = new UdpImageClient(this);
connect(widget, SIGNAL(receiveImage(QImage)), client, SLOT(append(QImage)));                
client->setServerIP("127.0.0.1");
client->setServerPort(6000);
client->start();

//服务端类使用方法
TcpImageServer *tcpServer = new TcpImageServer(this);
UdpImageServer *tcpServer = new UdpImageServer(this);
connect(tcpServer, SIGNAL(receiveImage(QString, QImage)), this, SLOT(receiveImage(QString, QImage)));