20200316 Websocket introduction of code release

Code publishing system

Course Outline

全栈项目，课时周期一周(周一到周五)

两天:单独的小知识点补充

三天:用前面的小知识点评凑出我们的代码发布功能

需要掌握:知识点、开发思路

Early knowledge review

ajax operation

异步提交,局部刷新

​```python
$.ajax({
	url: '',  // 控制提交地址
	type: '',  //控制提交方式
	data: '',  // 控制提交数据
	dataType:'JSON',
	success:function(args){
		//异步回调函数
        res = JSON.parse(args)
	}
})

Django后端如果返回的是HTTPresponse对象, json格式字符串,那么不会自动转换
如果是JSONResponse对象,那么会自动转换
配置该参数之后,无论什么情况都会讲后端json字符串装换成前端自定义对象数据

所以后面写ajax的时候,建议将该参数带上

Each Django application can have its own url.py, views.py (it can also be divided into different py files according to different functions), template template folder (the search order of templates is according to the app registered in the configuration file (The order is from top to bottom)

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'app01.apps.App01Config',
]

queue

Data structure and algorithm: required

链表: 判断链表是否有环 约瑟夫问题

算法:
	快排,堆排序原理,树形结构

• Queue: first-in first-out
• Stack: First in, late out

from django.test import TestCase

# Create your tests here.
import  queue


# 创建一个队列
q = queue.Queue()


# 往队列中添加数据
q.put(111)
q.put(222)

# 去队列中取数据
v1 = q.get()
v2 = q.get()
try:
    v4 = q.get(timeout=3)  # 没有数据 等待10s之后才报错  queue.Empty
    print(v4)
except queue.Empty as e:
    pass
# v3 = q.get()  # 一旦获取不到数据 默认是阻塞
# q.get_nowait()  # 一旦没有数据立马报错

print(v1,v2)

Thinking

Based on ajax and queue, can we achieve the feeling that the server pushes messages to the client

The server maintains a queue for each client browser, and then the browser requests data from the server through ajax (requests the data in the queue corresponding to each client Liu basketball) .If there is no data, it blocks in place (the front-end browser checks the network Status panding)

As long as a client sends a message to the server, the server sends the message to each queue

Recursive

# python中最大递归深度 997 998 官网提供的答案是1000
"""python中是没有尾递归优化的"""
def func():
  func()
func()

# 在前端js中 是没有递归一说的 函数自己调用自己是可以的 属于一种事件处理完全OK
function func(){
  $.ajax({
  url:'',  // 控制提交地址
  type:'',  // 控制提交方式
  data:{},  // 控制提交数据
  dataType:'JSON',
  success:function(args){
    func()
  }
})
}
func()

modelform verification component

It is an enhanced version of the forms component, and the functions are also three major blocks

Verify data, render tags, display error messages

Introduction

一般的IT互联网公司都会有一套自己的代码发布系统

目前来说大部分代码是基于运维jenkins来实现(shell脚本),其实也有公司自己定制自己的代码发布系统，定制的时候可以基于很多其他的技术点(saltstack、java、PHP、python系统)

我们的代码发布项目虽然是给运维或者测试用的，但是设计到的技术点基本全部都是python相关，我们注重的是开发逻辑

WebSocket 协议在2008年诞生，2011年成为国际标准。现在所有浏览器都已经支持了。WebSocket 的最大特点就是，服务器可以主动向客户端推送信息，客户端也可以主动向服务器发送信息，是真正的双向平等对话。
HTTP 有 1.1 和 1.0 之说，也就是所谓的 keep-alive ，把多个 HTTP 请求合并为一个，但是 Websocket 其实是一个新协议，跟 HTTP 协议基本没有关系，只是为了兼容现有浏览器，所以在握手阶段使用了 HTTP 。

The server actively pushes messages to the client

So far, the web projects we have written are basically based on the HTTP protocol between the server and the client to achieve data interaction

HTTP协议四大特性
	基于请求响应
	基于TCP/IP作用于应用层的协议
	无状态
	无连接

无连接: 我请求你,你给我响应,之后我俩就没关系了,并且都是浏览器主动请求服务端,服务端被动做出响应


长连接: websocket (互联网套接字) 相当于是HTTP协议的一个大的补丁,他支持长连接

Implementation idea: Let the browser secretly want the server to send a request (ajax) request data: polling / long polling (pseudo implementation)

The currently hot websocket, after the link is created, it is not disconnected (real realization)

Application scenario

• Draft big screen vote
• Task execution process

gojs plugin

Front-end plugins, dynamically generated charts, flowcharts, organizational charts, etc.

paramiko module

Similar to xshell remote connection server and operation, the bottom layer uses SSH connection

It will also encapsulate a class to make the operation more convenient (just copy the code)

gitpython module

Operate Git through python code

It will also encapsulate a class to make the operation more convenient (just copy the code)

Pseudo-implementation

• Polling (low efficiency, basically not used)

  轮询既轮番询问
      让浏览器定时(例如每隔5s发送一次)通过ajax偷偷的向服务端发送请求数据
  
  
  不足之处:
  	消息延迟,请求次数过多,损耗资源严重
  	
  

• Long polling (good compatibility)

  服务端给每个客户端创建一个队列,让浏览器通过ajax发送请求,请求各自队列中的数据,如果没有数据则会阻塞,但不会一直阻塞,利用timeout参数加异常处理的形式,最多阻塞30s返回,浏览器判断是否有数据,没有则继续发送(目前网页版的wx和qq用的还是这个原理)
  
  相对于轮询
  	没有消息延迟的
  	请求次数降低了,资源损耗偏小
  

  A simple version of group chat function based on long polling

  • The user visits the homepage with a unique identifier

Code

Long polling to achieve a simple version of group chat function

# 长轮询实现聊天室功能
url(r'^home/$',views.home),
url(r'^send_msg/$',views.send_msg),
url(r'^get_msg/$',views.get_msg)

views.py

from django.shortcuts import render, HttpResponse, redirect
from django.http import JsonResponse
# Create your views here.
import queue
# 定义一个存储用户队列的字典
q_dict = {}   # {'Jason':队列}

def home(request):
    # 获取自定义的唯一标识
    name = request.GET.get('name')   # 定义请求url,携带用于标识用户的name
    # 给每个用户都生成一个对应的队列对象
    q_dict[name] = queue.Queue()

    return render(request,'home.html',locals())


def send_msg(request):
    if request.method == "POST":
        # 获取用户提交的内容
        msg = request.POST.get('msg')
        # 循环获取所有客户端浏览器对应的队列对象
        for q in q_dict.values():
            q.put(msg)  # 添加到所有的用户队列中
    return HttpResponse('ok')


def get_msg(request):
    # 获取用户的当前name, 去用户自己对应的队列中获取数据
    name = request.GET.get('name')
    # 获取全局字典中的对应队列
    q = q_dict.get(name)
    # ajax交互一般用的都是字典格式
    back_dic = {'status':True,'msg':''}
    # 异常处理
    try:
        data = q.get(timeout=10)  # 等10s
        # 返回给前端页面数据 （ data为队列中的数据）
        back_dic['msg'] = data
    except queue.Empty as e:
        # 出错将status改为False
        back_dic['status'] = False

    return JsonResponse(back_dic)

html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>聊天室</title>
    <script src="https://cdn.bootcss.com/jquery/3.4.1/jquery.min.js"></script>
</head>
<body>

<h1>聊天室:{{ name }}</h1>
<div>
    <input type="text" id="i1">
    <button onclick="sendMsg()">发送</button>
</div>

<h1>聊天纪录</h1>
<div class="record">

</div>

<script>
    // 向后端url发送前端用户输入的数据
    function sendMsg() {
        $.ajax({
            url:'/send_msg/',
            type:'post',
            data:{'msg':$('#i1').val(),'csrfmiddlewaretoken':'{{ csrf_token }}'},
            success:function (args) {
            }
        })
    }
    //ajax请求向后端获取数据,跟队列要  (在页面加载完毕时触发)
    function getMsg() {
        $.ajax({
            url:'/get_msg/',
            type:'get',
            dataType:'JSON',
            // 携带唯一标识
            data:{'name':'{{ name }}','csrfmiddlewaretoken':'{{ csrf_token }}'},
            success:function (args) {
                //判断是否有数据回来了 , 如果没有,则继续调用自身
                if (args.status){
                    // 有消息进行消息的展示
                    // 将消息通过DOM操作展示到前端页面
                        // 1.创建标签
                    var pEle = $('<p>');

                        //2 添加文本内容
                    pEle.text(args.msg);

                        //3 将标签添加到页面对应的位置
                    $('.record').append(pEle)
                }
                // 没有消息就继续调用自己
                getMsg()
            }
        })
    }

    // 页面加载完毕之后就应该触发循环
    $(function () {
        getMsg()
    })
</script>

</body>
</html>

True realization

websocket (supported by all major browsers)

Introduction

网络协议
	http	: 不加密传输
	https	: 加密传输
	上面的两个协议都是短链接
	
	websocket : 加密传输
		浏览器与服务端建立链接之后默认不断开,两端都可以基于该链接收发消息
		websocket协议的诞生,真正意义上实现了服务端给客户端推送消息

Internal principle

websocket内部原理大致可以分为两部分

1.握手环节:验证服务端是否支持websocket协议
	- 浏览器访问服务端
    	浏览器会自动生成一个随机字符串，然后将该字符串自己保留一份给服务端也发送一份，这一阶段的数据交互是基于HTTP协议的(该随机字符串是放在请求头中的)
        
        GET / HTTP/1.1
        Host: 127.0.0.1:8080
        Connection: Upgrade
        ...
        Sec-WebSocket-Key: kQHq6MzLH7Xm1rSsAyiD8g==
        Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
    
    
    - 浏览器和服务端手上都有随机字符串
    	服务端从请求头中获取随机字符串之后，会先拿该字符串跟magic string(固定的随机字符串)做字符串的拼接，会对拼接之后的数据进行加密处理(sha1/base64)  于此同时浏览器那边也会做相同的操作
        
    - 服务端将处理好的随机字符串再次发送给浏览器(响应头)
    
    - 浏览器会对比自己生成的随机字符串和服务端发送的随机字符串是否一致,如果一致说明支持websocket一致,如果不支持则会报错不支持
    
    
    
2.收发数据:密文传输,数据解密的问题
    ps: 
        1.基于网络传输,数据都是二进制格式(python中的bytes类型)
        2.单位换算
    - 数据解密
    	1.先读取第二个字节的后七位二进制数(payload)
		
		2.根据payload不同做不同的处理
			=127:继续读8个字节
			=126:继续读2个字节
			<=125:不再往后读取
		
		3.往后读取固定长度的4个字节的数据(masking-key)
			根据该值计算出真实数据
"""
# 这些原理了解即可 关键需要说出几个关键字
	握手环节
  magic string  sha1/base64
  127、126、125
  payload masking-key
    
    	

Code verification

import socket
import hashlib
import base64

# 正常的socket代码
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# 防止mac/linux在重启的时候 报端口被占用的错
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)


sock.bind(('127.0.0.1', 8080))
sock.listen(5)

conn, address = sock.accept()
data = conn.recv(1024)  # 获取客户端发送的消息

def get_headers(data):
    """
    将请求头格式化成字典
    :param data:
    :return:
    """
    header_dict = {}
    data = str(data, encoding='utf-8')

    header, body = data.split('\r\n\r\n', 1)
    header_list = header.split('\r\n')
    for i in range(0, len(header_list)):
        if i == 0:
            if len(header_list[i].split(' ')) == 3:
                header_dict['method'], header_dict['url'], header_dict['protocol'] = header_list[i].split(' ')
        else:
            k, v = header_list[i].split(':', 1)
            header_dict[k] = v.strip()
    return header_dict

def get_data(info):
    """
    按照websocket解密规则针对不同的数字进行不同的解密处理
    :param info:
    :return:
    """
    payload_len = info[1] & 127
    if payload_len == 126:
        extend_payload_len = info[2:4]
        mask = info[4:8]
        decoded = info[8:]
    elif payload_len == 127:
        extend_payload_len = info[2:10]
        mask = info[10:14]
        decoded = info[14:]
    else:
        extend_payload_len = None
        mask = info[2:6]
        decoded = info[6:]

    bytes_list = bytearray()
    for i in range(len(decoded)):
        chunk = decoded[i] ^ mask[i % 4]
        bytes_list.append(chunk)
    body = str(bytes_list, encoding='utf-8')

    return body

header_dict = get_headers(data)  # 将一大堆请求头转换成字典数据  类似于wsgiref模块
client_random_string = header_dict['Sec-WebSocket-Key']  # 获取浏览器发送过来的随机字符串

magic_string = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11'  # 全球共用的随机字符串 一个都不能写错
value = client_random_string + magic_string  # 拼接

ac = base64.b64encode(hashlib.sha1(value.encode('utf-8')).digest())  # 加密处理

# 响应头
tpl = "HTTP/1.1 101 Switching Protocols\r\n" \
      "Upgrade:websocket\r\n" \
      "Connection: Upgrade\r\n" \
      "Sec-WebSocket-Accept: %s\r\n" \
      "WebSocket-Location: ws://127.0.0.1:8080\r\n\r\n"
response_str = tpl %ac.decode('utf-8')  # 处理到响应头中

# 将随机字符串给浏览器返回回去
conn.send(bytes(response_str, encoding='utf-8'))


# 收发数据
while True:
      data = conn.recv(1024)
      # print(data)

      value = get_data(data)

      print(value)

<script>
    var ws = new WebSocket('ws://127.0.0.1:8080/');
    // 这一行代码干了很多事
    // 1.自动生成随机字符串
    // 2.自动处理随机字符串 magic string  sha1/base64
    // 3.自动比对
</script>

This is an internal principle. Our actual production does not need to write these codes, just use the packaged module directly

Django implements websocket

Not all back-end frameworks support websocket

python三大主流web框架对websocket的支持

- Django: 
	默认不支持
	第三方模块: channels
	
- flask:
	默认不支持
	第三方模块: geventwebsocket
	
- tornado:
	默认就支持

channels

If you want to develop websocket related functions in Django, you need to install the module

python解释器 3.6
django版本 1.11.10
pip install channels==2.3

Precautions:

• Don't install the latest version of channels directly, it may automatically upgrade your Django version to the latest version
• The Python interpreter environment is recommended to use 3.6 (the official website said: 3.5 may have problems, 3.7 may also have problems ... the specific problem did not say ==)

The channels have already encapsulated all the processes demonstrated by the above code for you (handshake encryption and decryption)

Backend use

1. Register channels

Register in the settings file

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'app01.apps.App01Config',
    'channels'
]

启动django项目会报错CommandError: You have not set ASGI_APPLICATION, which is needed to run the server. Need to be configured

2. Configuration parameters

settings file configuration

ASGI_APPLICATION = 's12_day01.routing.application'

# ASGI_APPLICATION = '项目名同名的文件名称.routing.py文件名.application变量名'

3. Create the file

Create a routing.pyfile and write a fixed code in a folder with the same name as the project name

from channels.routing import ProtocolTypeRouter,URLRouter

application = ProtocolTypeRouter({
    'websocket':URLRouter([
        
        # 这里些websocket相关的url与视图函数对应关系
        
    ])
})

After the above three steps configuration is completed, start django again, it will support both http protocol and websocket protocol

Afterwards, the correspondence between http url and view function is still written in the original urls.py

The corresponding relationship between the URL of websocket and the view function is written in routing.py

Front-end use

<script>
  var ws = new WebSocket('ws://127.0.0.1:8080/')
  
  // 发送消息
  ws.send('你好啊')
</script>