Introduction to Golang Web (2): How to implement a RESTful routing

Summary

In the last article , we talked about how to implement an Http server in Golang. But in the end we can find that although it DefaultServeMuxcan be used as a routing distribution function, its function is also imperfect.

It DefaultServeMuxis impossible to implement a RESTfulstyle API by doing route distribution. We have no way to define the method required by the request, nor can APIwe add queryparameters to the path . Secondly, we also hope to make routing lookup more efficient.

So in this article, we will analyze httprouterthis package and study how he implements the functions we mentioned above from the source level. And, for the most important prefix tree in this package , this article will be explained in a combination of graphics and text.

1 use

We also start with how to use it and study from top to bottom httprouter. Let's take a look at the small example in the official documentation:

package main

import (
    "fmt"
    "net/http"
    "log"

    "github.com/julienschmidt/httprouter"
)

func Index(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
    fmt.Fprint(w, "Welcome!\n")
}

func Hello(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
    fmt.Fprintf(w, "hello, %s!\n", ps.ByName("name"))
}

func main() {
    router := httprouter.New()
    router.GET("/", Index)
    router.GET("/hello/:name", Hello)

    log.Fatal(http.ListenAndServe(":8080", router))
}
复制代码

In fact, we can find that the approach here net/httpis similar to the approach of using Golang's own package. All register the corresponding URI and function first, in other words, match the route with the processor.

When registering, use the router.XXXmethod to register the corresponding method, for example GET, POSTetc.

After registration, use to http.ListenAndServestart monitoring.

As for why, we will introduce it in detail in later chapters, now we only need to understand the practice first.

2 Create

Let's take a look at the first line of code. We have defined and declared one Router. Let's take a look at Routerthe structure of this, and omit other attributes not related to this article:

type Router struct {
	//这是前缀树，记录了相应的路由
	trees map[string]*node
	
	//记录了参数的最大数目
	maxParams  uint16

}
复制代码

After creating this Routerstructure, we use router.XXXmethods to register routes. Continue to see how the route is registered:

func (r *Router) GET(path string, handle Handle) {
	r.Handle(http.MethodGet, path, handle)
}

func (r *Router) POST(path string, handle Handle) {
	r.Handle(http.MethodPost, path, handle)
}

...
复制代码

There is a long list of methods here, they are all the same, call

r.Handle(http.MethodPost, path, handle)
复制代码

this way. Let's take another look:

func (r *Router) Handle(method, path string, handle Handle) {
	...
	if r.trees == nil {
		r.trees = make(map[string]*node)
	}

	root := r.trees[method]
	if root == nil {
		root = new(node)
		r.trees[method] = root

		r.globalAllowed = r.allowed("*", "")
	}

	root.addRoute(path, handle)
	...
}
复制代码

In this method, many details are also omitted. We only focus on what is relevant to this article. We can see that in this method, if treenot initialized, the prefix tree is initialized first .

Then we noticed that this tree is a mapstructure. In other words, a method corresponds to a tree. Then, corresponding to this tree, call the addRoutemethod URIand Handlesave the corresponding .

3 Prefix tree

3.1 Definition

Also known as the word search tree, the Trie tree is a tree structure and a variant of the hash tree. The typical application is to count, sort and save a large number of strings (but not limited to strings), so it is often used by search engine systems for text word frequency statistics. Its advantages are: use the common prefix of the string to reduce query time, minimize unnecessary string comparison, and query efficiency is higher than the hash tree.

Simply put, what you are looking for, as long as you follow a certain path of this tree, you can find it.

For example, in the search engine, you enter a Cai :

He will have these associations, which can also be understood as a prefix tree.

Another example:

The GETprefix tree of this method contains the following routes:

• /wow/awesome
• /test
• /hello/world
• /hello/china
• /hello/chinese

At this point, you should understand that in the process of building this tree, any two nodes, as long as they have the same prefix, the same part will be merged into one node .

3.2 Graphical construction

The addRoutemethod mentioned above is the method of inserting this prefix tree. Assuming that the number is empty, here I intend to illustrate the construction of this tree graphically.

Suppose the three routes we need to insert are:

• /hello/world
• /hello/china
• /hello/chinese

(1) Insert/hello/world

Because the tree is empty at this time, you can insert it directly:

(2) Insert/hello/china

In this case, we found /hello/worldand /hello/chinahave the same prefix /hello/.

Then, the original /hello/worldnode should be split first , and then the node to be inserted should be /hello/chinacut off as /hello/worldthe child node.

(3) Insert/hello/chinese

At this point, we need to insert /hello/chinese, but found, /hello/chineseand the node /hello/have a common prefix /hello/, so we went to see /hello/the child nodes of this node.

Note that there is an attribute in the node, called indices. It records the first letter of the child node of this node, which is convenient for us to find. For example, this /hello/node has his indicesvalue wc. The node we want to insert is the first letter /hello/chineseafter removing the common prefix , so we enter this node.chinesecchina

At this time, did you find that the situation returned to the /hello/chinasituation when we first inserted it? At that time, the public prefix was /hello/, and now the public prefix is chin.

Therefore, we also chincut it out as a node, which will be aa child of this node. And, it is also regarded eseas a child node.

3.3 Summary of the construction algorithm

At this point, the construction is over. Let's summarize the algorithm.

The specific annotated code will be given at the end of this article , if you want to learn more, you can check it yourself. First understand the process here:

(1) If the tree is empty, insert directly
(2) Otherwise, find whether the current node URIhas a common prefix with the one to be inserted (3) If there is no common prefix, insert directly (4) If there is a common prefix, judge Do you need to split the current node
(5) If you need to split, use the common part as the parent node, and the rest as child nodes
(6) If you do not need to split, look for the child nodes with the same prefix
(7) If they are the same, skip to (4)
(8) If there is no prefix with the same, directly insert
(9) at the last node and put this route corresponding toHandle

But here, some students have to ask: Why is the route here without parameters?

In fact, as long as you understand the above process, the parameters are the same. The logic is this: before each insertion, the path of the node to be inserted will be scanned for parameters (that is, whether there is a scan /or not *). If there are parameters, set the wildChildattribute of the current node to true, and then set the parameter part to a new child node .

4 Monitor

After talking about routing registration, let's talk about routing monitoring.

In the content of the previous article , we mentioned this:

type serverHandler struct {
	srv *Server
}

func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
	handler := sh.srv.Handler
	if handler == nil {
		handler = DefaultServeMux
	}
	if req.RequestURI == "*" && req.Method == "OPTIONS" {
		handler = globalOptionsHandler{}
	}
	handler.ServeHTTP(rw, req)
}
复制代码

At that time we mentioned that if we do not pass in any Handlemethod, Golang will use the default DefaultServeMuxmethod to process the request. And now we have passed in router, so it will be used routerto process the request.

Therefore, routerthe ServeHTTPmethod is also realized . Let's take a look (some steps are also omitted):

func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	...
	path := req.URL.Path

	if root := r.trees[req.Method]; root != nil {
		if handle, ps, tsr := root.getValue(path, r.getParams); handle != nil {
			if ps != nil {
				handle(w, req, *ps)
				r.putParams(ps)
			} else {
				handle(w, req, nil)
			}
			return
		} 
	}
    ...
    // Handle 404
	if r.NotFound != nil {
		r.NotFound.ServeHTTP(w, req)
	} else {
		http.NotFound(w, req)
	}
}
复制代码

Here, we select the prefix tree corresponding to the request method and call the method.getValue

Briefly explain this method: in this method, it will continuously match the current path and the node pathuntil it finds the Handlemethod corresponding to this route .

Note that during this period, if the route is RESTful and contains parameters in the route, it will be saved in Param. The Paramstructure here is as follows:

type Param struct {
	Key   string
	Value string
}
复制代码

If no corresponding route is found, the following 404 method is called.

5 Processing

At this point, it is almost the same as before.

After obtaining the corresponding route Handle, call this function.

The only difference from the previous net/httppackage Handleris that here Handle, the parameters obtained from the API are encapsulated.

type Handle func(http.ResponseWriter, *http.Request, Params)
复制代码

6 at the end

Thank you for seeing here ~

At this point, the introduction of httprouter is completed, the most critical is the construction of the prefix tree. In the above, I used a combination of graphics and text to simulate the construction process of a prefix tree. I hope you can understand what the prefix tree is. Of course, if you still have questions, you can leave a message or communicate with me on WeChat ~

Of course, if you are not satisfied with this, you can see attached to the back of the catalog, there is a prefix tree full code comments .

Of course, the author is just getting started. Therefore, there may be many omissions. If there are any explanations that are not in place during the reading process, or if there is a deviation in your understanding, please leave a message to correct me.

Thanks again ~

PS: If you have other questions, you can also find the author on the public account. In addition, all articles will be updated on the public account at the first time. Welcome to find the author to play ~

7 Source code reading

7.1 Tree structure

type node struct {
	
	path      string    //当前结点的URI
	indices   string    //子结点的首字母
	wildChild bool      //子节点是否为参数结点
	nType     nodeType  //结点类型
	priority  uint32    //权重
	children  []*node   //子节点
	handle    Handle    //处理器
}
复制代码

7.2 addRoute

func (n *node) addRoute(path string, handle Handle) {

	fullPath := path
	n.priority++

	// 如果这是个空树，那么直接插入
	if len(n.path) == 0 && len(n.indices) == 0 {

		//这个方法其实是在n这个结点插入path，但是会处理参数
		//详细实现在后文会给出
		n.insertChild(path, fullPath, handle)
		n.nType = root
		return
	}

	//设置一个flag
walk:
	for {
		// 找到当前结点path和要插入的path中最长的前缀
		// i为第一位不相同的下标
		i := longestCommonPrefix(path, n.path)

		// 此时相同的部分比这个结点记录的path短
		// 也就是说需要把当前的结点分裂开
		if i < len(n.path) {
			child := node{

				// 把不相同的部分设置为一个切片，作为子节点
				path:      n.path[i:],
				wildChild: n.wildChild,
				nType:     static,
				indices:   n.indices,
				children:  n.children,
				handle:    n.handle,
				priority:  n.priority - 1,
			}

			// 将新的结点作为这个结点的子节点
			n.children = []*node{&child}
			// 把这个结点的首字母加入indices中
			// 目的是查找更快
			n.indices = string([]byte{n.path[i]})
			n.path = path[:i]
			n.handle = nil
			n.wildChild = false
		}

		// 此时相同的部分只占了新URI的一部分
		// 所以把path后面不相同的部分要设置成一个新的结点
		if i < len(path) {
			path = path[i:]

			// 此时如果n的子节点是带参数的
			if n.wildChild {
				n = n.children[0]
				n.priority++

				// 判断是否会不合法
				if len(path) >= len(n.path) && n.path == path[:len(n.path)] &&
					n.nType != catchAll &&
					(len(n.path) >= len(path) || path[len(n.path)] == '/') {
					continue walk
				} else {
					pathSeg := path
					if n.nType != catchAll {
						pathSeg = strings.SplitN(pathSeg, "/", 2)[0]
					}
					prefix := fullPath[:strings.Index(fullPath, pathSeg)] + n.path
					panic("'" + pathSeg +
						"' in new path '" + fullPath +
						"' conflicts with existing wildcard '" + n.path +
						"' in existing prefix '" + prefix +
						"'")
				}
			}

			// 把截取的path的第一位记录下来
			idxc := path[0]

			// 如果此时n的子节点是带参数的
			if n.nType == param && idxc == '/' && len(n.children) == 1 {
				n = n.children[0]
				n.priority++
				continue walk
			}

			// 这一步是检查拆分出的path，是否应该被合并入子节点中
			// 具体例子可看上文中的图解
			// 如果是这样的话，把这个子节点设置为n，然后开始一轮新的循环
			for i, c := range []byte(n.indices) {
				if c == idxc {
					// 这一部分是为了把权重更高的首字符调整到前面
					i = n.incrementChildPrio(i)
					n = n.children[i]
					continue walk
				}
			}

			// 如果这个结点不用被合并
			if idxc != ':' && idxc != '*' {
				// 把这个结点的首字母也加入n的indices中
				n.indices += string([]byte{idxc})
				child := &node{}
				n.children = append(n.children, child)
				n.incrementChildPrio(len(n.indices) - 1)
				// 新建一个结点
				n = child
			}
			// 对这个结点进行插入操作
			n.insertChild(path, fullPath, handle)
			return
		}

		// 直接插入到当前的结点
		if n.handle != nil {
			panic("a handle is already registered for path '" + fullPath + "'")
		}
		n.handle = handle
		return
	}
}
复制代码

7.3 insertChild

func (n *node) insertChild(path, fullPath string, handle Handle) {
	for {
		// 这个方法是用来找这个path是否含有参数的
		wildcard, i, valid := findWildcard(path)
		// 如果不含参数，直接跳出循环，看最后两行
		if i < 0 {
			break
		}

		// 条件校验
		if !valid {
			panic("only one wildcard per path segment is allowed, has: '" +
				wildcard + "' in path '" + fullPath + "'")
		}

		// 同样判断是否合法
		if len(wildcard) < 2 {
			panic("wildcards must be named with a non-empty name in path '" + fullPath + "'")
		}

		if len(n.children) > 0 {
			panic("wildcard segment '" + wildcard +
				"' conflicts with existing children in path '" + fullPath + "'")
		}

		// 如果参数的第一位是`:`，则说明这是一个参数类型
		if wildcard[0] == ':' {
			if i > 0 {
				// 把当前的path设置为参数之前的那部分
				n.path = path[:i]
				// 准备把参数后面的部分作为一个新的结点
				path = path[i:]
			}

			//然后把参数部分作为新的结点
			n.wildChild = true
			child := &node{
				nType: param,
				path:  wildcard,
			}
			n.children = []*node{child}
			n = child
			n.priority++

			// 这里的意思是，path在参数后面还没有结束
			if len(wildcard) < len(path) {
				// 把参数后面那部分再分出一个结点，continue继续处理
				path = path[len(wildcard):]
				child := &node{
					priority: 1,
				}
				n.children = []*node{child}
				n = child
				continue
			}

			// 把处理器设置进去
			n.handle = handle
			return

		} else { // 另外一种情况
			if i+len(wildcard) != len(path) {
				panic("catch-all routes are only allowed at the end of the path in path '" + fullPath + "'")
			}

			if len(n.path) > 0 && n.path[len(n.path)-1] == '/' {
				panic("catch-all conflicts with existing handle for the path segment root in path '" + fullPath + "'")
			}

			// 判断在这之前有没有一个/
			i--
			if path[i] != '/' {
				panic("no / before catch-all in path '" + fullPath + "'")
			}

			n.path = path[:i]

			// 设置一个catchAll类型的子节点
			child := &node{
				wildChild: true,
				nType:     catchAll,
			}
			n.children = []*node{child}
			n.indices = string('/')
			n = child
			n.priority++

			// 把后面的参数部分设置为新节点
			child = &node{
				path:     path[i:],
				nType:    catchAll,
				handle:   handle,
				priority: 1,
			}
			n.children = []*node{child}

			return
		}
	}

	// 对应最开头的部分，如果这个path里面没有参数，直接设置
	n.path = path
	n.handle = handle
}
复制代码

The most critical methods are all over here. Let's applaud you who see here!

This part will be difficult to understand, and may need to be read several times.

If there is still a place that is difficult to understand, please leave a message to exchange, or directly come to the public number to find me ~