In the previous article , we used PCAP to build a local software hub (Hub). Considering the long-distance inter-workshop debugging, it is necessary to use Tcp connection to construct an Ethernet tunnel, so that the debugging equipment between two workshops can be virtually connected to a Hub. Of course, we can use QTcpSocket to realize the connection, or try the QSslSocket secure socket, just use the certificate generated in the previous article to play with the Ssl connection.
For the complete project, refer to the Git link in the previous article .
The schematic diagram of a debugging workstation with a remote Ssl tunnel is as follows:
1. Point-to-point TCP connection
We hope that the PCAPHub processes on the two debugging workstations can be connected through a peer-to-peer channel. Workstation A is in monitor mode and workstation B is in client mode.
In this way, a pair of TCP connections is equivalent to a logical network cable, and is in an equal relationship with each local network port. In the figure above, all remote MAC addresses will appear in the "TCP tunnel peer connection MAC column".
2. Build an SSL server
In Qt6, the QSslServer class is set, and the Ssl server can be directly constructed. But in Qt5, there is no such class. In order to be compatible with Qt5, it is necessary to overload QTcpServer and construct a SslServer
//"sslserver.h"
class SSLServer : public QTcpServer
{
Q_OBJECT
public:
explicit SSLServer(QObject *parent = nullptr);
protected:
void incomingConnection(qintptr socketDescriptor) override;
signals:
void sig_newClient(qintptr socketDescriptor);
};
//CPP
void SSLServer::incomingConnection(qintptr socketDescriptor)
{
emit sig_newClient(socketDescriptor);
}
Then, to respond in the TcpTunnel class derived from QObject, sig_newClient will pump the socket descriptor to tcpTunnel. In the source code, you will find that these signals and Cao are running in separate threads, not the main UI thread. The tcpTunnel class is used to instantiate objects working on QThread, so as to use QThread's independent message loop to drive the flow of signals and events.
class tcpTunnel : public QObject
{
//...
//Tunnel
protected:
SSLServer * m_svr = nullptr;
QTcpSocket * m_sock = nullptr;
protected slots:
void slot_new_connection(qintptr socketDescriptor);
};
//New Connections
void tcpTunnel::slot_new_connection(qintptr socketDescriptor)
{
QTcpSocket * sock = m_bSSL? new QSslSocket(this):new QTcpSocket(this);
if (sock->setSocketDescriptor(socketDescriptor)) {
connect(sock,&QTcpSocket::readyRead,this,&tcpTunnel::slot_read_sock);
//SSL Handshake
QSslSocket * sslsock = qobject_cast<QSslSocket*>(sock);
if (sslsock)
{
QString strCerPath = ":/certs/svr_cert.pem";
QString strPkPath = ":/certs/svr_privkey.pem";
sslsock->setLocalCertificate(strCerPath);
sslsock->setPrivateKey(strPkPath);
sslsock->startServerEncryption();
}
}
}
The following points need to be explained here:
- QSslSocket and QTcpSocket have a highly consistent state machine, that is, the behavior of the state() function is highly consistent. Because of this, the SSL function can be turned on or off through a simple mark on the interface.
- As a server socket, certificates are specified before the handshake. It is problematic to use the example certificate directly embedded in the resource. Because the server address of the sample certificate is 127.0.0.1, which is different from the real environment, the client connection will fail. In this example, the client ignores the address mismatch error, which is bad behavior. In the production environment, it is still necessary to prepare the appropriate certificate according to the specific situation.
With the above logic, tcpTunnel can respond to the message and accept the connection to start handshaking when the client arrives.
3 The client initiates a connection
In the slot function of the tcpTunnel class, a connection to the server will be initiated. What needs to be noted here is that it is necessary to respond to the QSslSocket::sslErrors signal in time, and handle several types of certificate errors, so that the connection can still be established even if the certificate contains errors . The associated error is:
- QSslError::CertificateUntrusted untrusted certificate
- QSslError::CertificateNotYetValid certificate not yet valid (future moment)
- QSslError::CertificateExpired expired certificate
- QSslError::HostNameMismatch The server address of the certificate is different from the HostAddress of the current connection.
In addition, for Qt5, due to the overload of the signal QSslSocket::sslErrors, an explicit type constraint must be performed before the functional style connect can be performed. This problem has been avoided in Qt6.
//H
class tcpTunnel : public QObject
{
public slots:
void startWork(QString address, QString port, bool listen,bool ssl);
};
//CPP
void tcpTunnel::startWork(QString address, QString port, bool ssl)
{
if (m_bSSL)
{
QSslSocket * sslsock = new QSslSocket(this);
connect(sslsock,static_cast<void (QSslSocket::*)(const QList <QSslError> &)>(&QSslSocket::sslErrors),
[this,sslsock](const QList <QSslError> & err)->void
{
QList<QSslError> errIgnore;
foreach (QSslError e, err)
{
emit sig_message(tr("SSL Error %1:%2 .").arg((int)e.error()).arg(e.errorString()));
if (e.error()==QSslError::HostNameMismatch) errIgnore<<e;
else if (e.error()==QSslError::CertificateUntrusted) errIgnore<<e;
else if (e.error()==QSslError::CertificateNotYetValid) errIgnore<<e;
else if (e.error()==QSslError::CertificateExpired) errIgnore<<e;
}
sslsock->ignoreSslErrors(errIgnore);
});
m_sock = sslsock;
sslsock->connectToHostEncrypted(m_str_addr,m_n_port);
}
else
{
m_sock = new QTcpSocket(this);
m_sock->connectToHost(QHostAddress(m_str_addr),m_n_port);
}
connect(m_sock,&QTcpSocket::readyRead,this,&tcpTunnel::slot_read_sock);
emit sig_message(tr("connecting to: %1:%2").arg(m_str_addr).arg(m_n_port));
}
4. Ethernet on TCP protocol
In the previous article, the captured Ethernet data was stored in a ring cache. If Ethernet data is transmitted through tcp, a protocol for cutting packets needs to be designed. We do it the easiest way possible.
| Magic | length | data |
|---|---|---|
| 4Bytes | 2Bytes UShort | N1 Bytes |
| 4Bytes | 2Bytes UShort | N2 Bytes |
| 4Bytes | 2Bytes UShort | N3 Bytes |
| …… |
In this way, it is only necessary to detect and fetch data at the time of reception to complete the packetization. The code for sending the package is as follows:
quint64 rp = PCAPIO::pcap_recv_pos;
while (m_nTPos < rp) {
const int nPOS = m_nTPos % PCAPIO_BUFCNT;
const int fromID = PCAPIO::global_buffer[nPOS].from_id;
if (fromID !=TCPTUNID
&& PCAPIO::global_buffer[nPOS].len < 65536
&& PCAPIO::global_buffer[nPOS].len >0)
{
const unsigned char hd[] = {
0x18u,0x24u,0x7eu,0x69u};
const unsigned short len = PCAPIO::global_buffer[nPOS].len;
m_sock->write((const char *)hd,4);
m_sock->write((const char *)&len,2);
m_sock->write(PCAPIO::global_buffer[nPOS].data.constData(),len);
}
++m_nTPos;
}
The code for receiving packets is as follows:
class tcpTunnel : public QObject
{
void dealPack(const char * pack, const int len);
//Tunnel
private:
quint64 m_nTPos = 0;
QByteArray m_package_array;
protected slots:
void slot_read_sock();
};
void tcpTunnel::slot_read_sock()
{
QByteArray arrData = m_sock->readAll();
m_package_array.append(arrData);
while (static_cast<size_t>(m_package_array.size())>=6)
{
//检查Magic
int goodoff = 0;
while (!(m_package_array[0+goodoff]==(char)0x18 && m_package_array[1+goodoff]==(char)0x24
&&m_package_array[2+goodoff]==(char)0x7E &&m_package_array[3+goodoff]==(char)0x69 ))
{
++goodoff;
if (goodoff+3>= m_package_array.size())
break;
}
if (goodoff)
m_package_array.remove(0,goodoff);
if (m_package_array.size()< 6 )
break;
const unsigned short * ptrlen = (const unsigned short *)
(m_package_array.constData() + 4);
const unsigned short datalen = *ptrlen;
if (m_package_array.size()<datalen+6)
break;
const char * dptr = m_package_array.constData()+6;
//Enqueue入队
dealPack(dptr,datalen);
//清除当前包
m_package_array.remove(0,6+datalen);
}
}
5 Avoid nested storms
Through the above operations, it is theoretically possible to bring remote Ethernet packets to the local. However, since the TCP tunnel itself also runs on the Ethernet protocol, if PCAP captures the content of the TCP tunnel when capturing, then a traffic storm will occur.
How to avoid traffic storm? As long as the filter condition "not tcp port 12345" is appended to the user's condition before crawling, the storm can be avoided.
//2. Run Cap Thread on interface.
void recv_loop(QString itstr, QString filterStr, int id, int tcp_exlude,std::function<void (QString) > msg)
{
while (!pcap_stop)
{
pcap_t *handle = NULL;
//...
struct bpf_program filter;
//Combine Filter
QString ExFlt ;
if (tcp_exlude > 0 && tcp_exlude < 65536)
{
if (filterStr.trimmed().length())
ExFlt = "(" + filterStr.trimmed() + QString(") and not tcp port %1" ).arg(tcp_exlude);
else
ExFlt = QString("not tcp port %1" ).arg(tcp_exlude);
}
else
ExFlt = filterStr.trimmed();
//Compile Filter
if (ExFlt.size())
{
std::string filter_app = ExFlt.toStdString();
pcap_compile(handle, &filter, filter_app.c_str(), 0, net);
pcap_setfilter(handle, &filter);
}
}
}
}
Of course, sometimes this is not enough, and the packet capture conditions of each network port must be combined to jointly restrict. In principle, all tunnel-related traffic should be excluded from the PCAP conditions.
For example, if the TCP tunnel is connected through the ssh agent for -L or -R, then SSH should also be excluded. At the same time, since HUB is an inefficient broadcast, all continuous traffic like remote desktop 3389 should be turned off. Otherwise, it may compete with the device for bandwidth.
6 Flexible switching between HUB and switch modes
We only need to use a switch to control whether to play back the content belonging to all other network ports when the network card is playing back, so that the network port can work in the hub or switch mode.
struct tag_packages{
int from_id;//From which port
int to_id;//To which port
int len;
QByteArray data;
};
void recv_loop(QString itstr, QString filterStr, int id, int tcp_exlude,std::function<void (QString) > msg)
{
//...Dst Mac
const bool newDstMac = pcap_ports.contains(mac_dst);
if (newDstMac)
{
if(pcap_ports[mac_dst].dtmLastAck.msecsTo(dtm) <=CAP_FADE_MSECS)
dst_id = pcap_ports[mac_dst].curr_id;
}
}
//3. Run Send Thread on interface
void send_loop(QString itstr,int id, bool bSwitchMod,std::function<void (QString) > msg)
{
if (global_buffer[pos].from_id!=id &&
((!bSwitchMod)||(global_buffer[pos].to_id==id || global_buffer[pos].to_id==-1))
)
{
}
}
Using this strategy, when there are many network ports, PCAPHub can be flexibly controlled, making it work in a mixed mode that takes into account both efficiency and scope.
