Click above ↑ ↑ ↑ blue word [ protocol analysis and restoration ] to follow us
" Use python to implement the common TCP long connection framework in the protocol. "
If you analyze more protocols, you will find that many applications, especially game and IM applications, their protocols use long connections to maintain the connection between the client and the server. These long connections are usually carried by TCP.
The websocket we introduced before is a kind of long connection, but it is compatible with the HTTP protocol. Friends who are interested can read the previous article:
If we want to simulate the behavior of this client, according to the implementation of different application servers, some long connections are not necessary, but some long connections must be implemented. For example, in a recently analyzed application, although it mainly uses the HTTP protocol for interaction, it transmits some necessary information in the TCP long connection. If the long connection is not implemented, there will be a lot of information that cannot be processed.
In python, it is easy to implement the HTTP protocol. Of course, it is also easy to implement the TCP protocol. Its TCP implementation can use the socket library, but it should be noted that the hexadecimal data is usually transmitted in the TCP long connection, and the protocol is not Standard, you need to encapsulate the data format according to the protocol analysis results.
Here we take a protocol that uses TCP long-term connections as an example to give the TCP long-term connection framework of the protocol. You can refer to the implementation if you need it. Of course, the code is also extracted from the sample and is not complete.
My TCP long connection framework, first of all, the external packaging, initializes some parameters, such as the ip port and socket socket used by the long connection:
self.longip='im.langren001.com'
self.longport= 6656
self.threadLock = threading.Lock()
self.sockmain = socket.socket(socket.AF_INET, socket.SOCK_STREAM);
self.longlinktcpstart2()
tlonglink = threading.Thread(target=lrsuser.longlinktcpth2,name='mainlink_'+ self.playinfo['uid'], args=(self,))
tlonglink.start()
self.threadinfo.append(tlonglink)Two functions are called in this, one is the longlinktcpstart2 function, which is used to establish a socket connection, and realize the interaction when some connections are initially established, and the other is the longlinktcpth2 function, which is a thread to realize the sending and receiving of data in the connection . Generally speaking, these two can be implemented together, but in order to facilitate the handling of socket abnormal disconnection, it is divided into two functions.
The implementation of longlinktcpstart2 is as follows:
def longlinktcpstart2(self):
server_address = (self.longip, int(self.longport))
self.savelogs('longlinktcpstart2', 'Connecting to %s:%d.' % server_address)
self.sockmain.connect(server_address)
self.databuf = b''
message = genbaseinfo.genalive()
self.sockmain.sendall(message)
message = genbaseinfo.genfirstdata()
if len(message)==0:
self.savelogs('longlinktcpstart2', 'genfirstdata error ')
return False
self.sockmain.sendall(message)
self.longlinkcnt=2
cnt = 0
while (cnt < 2):
try:
buf = self.sockmain.recv(2048)
sz = len(buf)
self.savelogs('longlinktcpstart2', "recv data len "+str(sz) )
if sz > 0:
self.databuf +=buf
self.dealdatabuf()
if cnt == 0:
alivemsg = genbaseinfo.genalive()
self.sockmain.sendall(alivemsg)
self.savelogs('longlinktcpstart2', "sendalive")
regtime=int(round(time.time() * 1000))-random.randint(14400000,25200000)
regtime=regtime*1000
pcode = self.versionstr + '.0'
message = genbaseinfo.genseconddata()
if len(message) == 0:
self.savelogs('longlinktcpstart2', 'genseconddata error ')
return False
self.sockmain.sendall(message)
self.longlinkcnt = self.longlinkcnt + 1
elif cnt == 1:
pcode = self.versionstr + '.0'
message = genbaseinfo.genotherdata()
if len(message) == 0:
self.savelogs('longlinktcpstart2', 'genthirddata error ')
return False
self.sockmain.sendall(message)
self.longlinkcnt = self.longlinkcnt + 1
cnt = cnt + 1
else:
self.savelogs('longlinktcpstart2', 'recv data alive')
except: # socket.error
self.savelogs('longlinktcpstart2', 'socket error,do connect fail')
return False
return TrueThe genbaseinfo-related functions here can be ignored, they are used to generate the message data to be sent, and can be replaced with your own functions. The dealdatabuf function is used to process the received message data. These two must be implemented according to the specific protocol analysis. Note that the generated data for sending and the received data that needs to be processed need to be processed according to sixteen Hexadecimal processing is not described in detail here.
The thread longlinktcpth2 is a loop, the protocol does not exit, the loop does not end, the implementation is as follows:
def longlinktcpth2(self):
tmalive = 0;
r_inputs = set()
r_inputs.add(self.sockmain)
w_inputs = set()
w_inputs.add(self.sockmain)
e_inputs = set()
e_inputs.add(self.sockmain)
tm=int(round(time.time()))
self.savelogs('longlinktcpth2', 'enter' )
while (self.quitflag==0):
try:
r_list, w_list, e_list = select.select(r_inputs, w_inputs, e_inputs, 1)
for event in r_list:
try:
buf = event.recv(2048)
sz = len(buf)
self.savelogs('longlinktcpth2', "loop recv data len:"+ str(sz) )
if sz > 0:
self.databuf += buf
self.dealdatabuf()
alivemsg = genbaseinfo.genalive()
self.sockmain.sendall(alivemsg)
self.savelogs('longlinktcpth2', "sendalive")
else:
self.savelogs('longlinktcpth2', "远程断开连接,do reconnect")
r_inputs.clear()
time.sleep(3)
self.sockmain = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.longlinktcpstart2()
r_inputs = set()
r_inputs.add(self.sockmain)
w_inputs = set()
w_inputs.add(self.sockmain)
e_inputs = set()
e_inputs.add(self.sockmain)
except Exception as e:
self.savelogs('longlinktcpth2', str(e))
self.threadLock.acquire()
if (len(self.msglist) > 0):
msg = self.msglist.pop(0)
self.threadLock.release()
self.sockmain.sendall(msg)
self.savelogs('longlinktcpth2',"send a msg")
else:
self.threadLock.release()
tmnow=int(round(time.time()))
if tmnow-tm>30:
message = genbaseinfo.genotherdata()
if len(message) == 0:
self.savelogs('longlinktcpth2', 'genalivedata error ')
return False
self.sockmain.sendall(message)
self.savelogs('longlinktcpth2', "send alivemsg"+str(self.longlinkcnt))
self.longlinkcnt = self.longlinkcnt + 1 #这个要一条连接统一，不能乱，回头加锁
tm=tmnow
if len(w_list) > 0: # 产生了可写的事件，即连接完成
self.savelogs('longlinktcpth2',str(w_list))
w_inputs.clear() # 当连接完成之后，清除掉完成连接的socket
if len(e_list) > 0: # 产生了错误的事件，即连接错误
self.savelogs('longlinktcpth2', str(e_list))
e_inputs.clear() # 当连接有错误发生时，清除掉发生错误的socket
except OSError as e:
self.savelogs('longlinktcpth2', 'socket error,do reconnect')
time.sleep(3)
self.sockmain = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.longlinktcpstart2()
r_inputs = set()
r_inputs.add(self.sockmain)
w_inputs = set()
w_inputs.add(self.sockmain)
e_inputs = set()
e_inputs.add(self.sockmain)
self.savelogs('longlinktcpth2', 'leave')Since this code is mainly used on windows, the longlinktcpth2 thread is implemented using select instead of epoll. In the loop, the exception is handled. If an exception occurs and the connection is disconnected, call longlinktcpstart2 to reconnect without exiting the loop. The rest are consistent with longlinktcpstart2.
Since the TCP connection is a stream concept, data needs to be cached and spliced. This is the function of databuf in the above code to prevent incomplete or too much data received each time and facilitate subsequent processing. This is a qualified The self-sublimation of the coder's belief.
At this point, the implementation of the protocol TCP long connection under python is over. If you have similar needs, you can refer to the implementation.
You are welcome to come and sit and communicate more often.
Don't forget to click "Looking", "Like" and "Share"
The new rule, to receive tweets in time, you must first star the official account
Don't forget to star or you will miss out
Long press to follow and communicate all the time.