Web service based on HTTP protocol

Article directory

Web service based on HTTP protocol

1. Network service of HTTP protocol

The HTTP protocol is based on the TCP/IP protocol stack, and it is also a plain text-oriented protocol.

As long as you figure out what the HTTP request message (the header and body of the message) should contain, you can write a complete HTTP request message using any text compiler.

In this case, using net.Dialthe function directly is fine.

Using net/httpthe program entity in the code package, you can access the network service based on HTTP protocol more conveniently. The most convenient of these is to use http.Getfunctions.

1.1 Use `http.Get`functions to access web services of the HTTP protocol

package main

import (
	"fmt"
	"net/http"
)

func main() {
    
    
	url1 := "http://www.google.cn/"
	fmt.Printf("Send request to %q with method GET ... \n", url1)
	response1, err := http.Get(url1)
	if err != nil {
    
    
		fmt.Printf("request sending error: %v\n", err)
	}
	defer response1.Body.Close()
	line1 := response1.Proto + " " + response1.Status
	fmt.Printf("The first line of response: \n %s \n", line1)
}

http.GetThe function returns two result values:

The type of the first result value is *http.Responsethat it is a structured representation of the response content sent back to us by the web service.
The second result value is of type error. It represents errors that may occur during the process of creating and sending HTTP requests, and receiving and parsing HTTP responses.

http.GetThe default HTTP client will be used inside the function, and its Get method will be called to complete the function . The default client type is *http.Client, represented by the public variable DefaultClient.

1.2 Use the default client DefaultClient (type is `*http.Client`)

package main

import (
	"fmt"
	"net/http"
)

func main() {
    
    
	url1 := "http://www.google.cn/"
	fmt.Printf("Send request to %q with method GET ... \n", url1)
	// response1, err := http.Get(url1)
	response1, err := http.DefaultClient.Get(url1)
	if err != nil {
    
    
		fmt.Printf("request sending error: %v\n", err)
	}
	defer response1.Body.Close()
	line1 := response1.Proto + " " + response1.Status
	fmt.Printf("The first line of response: \n %s \n", line1)
}

Its primitive type ( http.Client) is available out of the box.

1.3 Using `http.Client`web services that access the HTTP protocol

package main

import (
	"fmt"
	"net/http"
)

func main() {
    
    
	url1 := "http://www.google.cn/"
	fmt.Printf("Send request to %q with method GET ... \n", url1)
	// response1, err := http.Get(url1)
	// response1, err := http.DefaultClient.Get(url1)
	var oneClient http.Client
	response1, err := oneClient.Get(url1)
	if err != nil {
    
    
		fmt.Printf("request sending error: %v\n", err)
	}
	defer response1.Body.Close()
	line1 := response1.Proto + " " + response1.Status
	fmt.Printf("The first line of response: \n %s \n", line1)
}

http.Clientis a struct type, and the fields it contains are public. The reason why the zero value of this type can still be used is because its fields either have the default value of the response, or its zero value can be used directly and represents a specific meaning.

2. `http.Client`The Transport field in

http.ClientThe Transport field in the type represents the operation process of sending an HTTP request to the web service and receiving an HTTP response from the web service.

The RoundTrip method of the Transport field implements all the steps required for a single HTTP transaction (or word interaction based on the HTTP protocol).

The Transport field is http.RoundTripan interface type, and it has a default value whose variable name is DefaultTransport. The actual type of DefaultTransport *http.Transportcan *http.Transportbe reused and is thread-safe.

If there is no explicit http.Clientassignment to the Transport field, the Client will directly use DefaultTransport.

http.ClientThe Timeout field in represents the timeout period of the aforementioned word HTTP transaction, it is time.Durationa type, and its zero value is available, which is used to indicate that no timeout period is set.

(1) `http.Transport`DialContext field in type

http.TransportType, use a type of value internally net.Dialer, and set the value of the Timeout field of the value to 30 seconds.

In other words, if the Dialer value has not established a network connection within 30 seconds, it will be judged as an operation timeout.

When the value of DefaultTransport is initialized, the DialContext method of such a Dialer value will be assigned to the former DialContext field:

var DefaultTransport RoundTripper = &Transport{
    
    
	Proxy: ProxyFromEnvironment,
	DialContext: defaultTransportDialContext(&net.Dialer{
    
    
		Timeout:   30 * time.Second,
		KeepAlive: 30 * time.Second,
	}),
	ForceAttemptHTTP2:     true,
	MaxIdleConns:          100,
	IdleConnTimeout:       90 * time.Second,
	TLSHandshakeTimeout:   10 * time.Second,
	ExpectContinueTimeout: 1 * time.Second,
}

func defaultTransportDialContext(dialer *net.Dialer) func(context.Context, string, string) (net.Conn, error) {
    
    
	return dialer.DialContext
}

Behind KeepAlive is a liveness detection mechanism for network connections (more precisely, TCP connections). Its value is used to indicate how often probe packets are sent. When the value is not greater than 0, it means that this mechanism is not enabled.

DefaultTransport will set the value of this field to 30 seconds.

(2) `http.Transport`Other fields in the type

Something about timeout operations

IdleConnTimeout: The meaning is how long after an idle connection should be closed.

DefaultTransport will set the value of this field to 90 seconds.

If the value is 0, it means that idle connections are not closed. Note that this may cause resource leaks.
ResponseHeaderTimeout: It means the maximum time from when the client completely submits the request to the operating system to when the response header is received from the operating system.

DefaultTransport does not set the value of this field.
ExpectContinueTimeout: It means, after the client submits the request header, wait for the longest time to receive the first response header.

DefaultTransport sets the value of this field to 1 second.

When the client wants to use the "POST" method of HTTP to send a large message body to the server, it can first ask the server by sending a request header containing "Expect: 100-continue". Are you willing to accept this large message body. This field is used to set the timeout period in this case.

Note that if the value of this field is not greater than 0, no matter how large the request body is, it will be sent immediately.
TLSHandshakeTimeout: TLS is the abbreviation of Transport Layer Security, which can be translated as Transport Layer Security. This field represents the timeout period of the handshake phase when the connection based on the TLS protocol is established.

DefaultTransport sets the value of this field to 10 seconds.

If the value is 0, it means that there is no limit on this value.

some IdleConnTimeoutfield values related to

MaxIdleConns: Used to control the maximum idle connection to access all hosts. If 0, no limit.

DefaultTransport sets MaxIdleConns to 100.

The MaxIdleConns field only limits the total number of idle connections.
MaxIdleConnsPerHost: Control the maximum number of idle connections for each network service accessed by the Transport value. If 0, the default value of 2 will be used, which is DefaultMaxIdleConnsPerHostrepresented by .

That is to say, by default, for each network service accessed by a certain Transport value, the number of idle connections can only be at most two.
MaxConnsPerHost: The maximum number of connections for each network service accessed for a certain Transport value, regardless of whether these connections are idle or not.

There is no default value for this field, and a value of zero means no limit.

MaxIdleConnsThe values of the two MaxIdleConnsPerHostfields related to the number of idle connections should be linked, so sometimes they need to be customized according to the actual situation, you can refer to the declaration of the DefaultTransport variable.

3. Why is there an idle connection

3.1 Generation of idle connections

The HTTP protocol has a request header called "Connection". In version 1.1 of the HTTP protocol, the value of this header is "keep-alive" by default.

In this case, network connections are persistent connections, they will still maintain connectivity after the current HTTP transaction is completed, so they can be reused.

The reusability of connection brings two possibilities:

One possibility is that a new HTTP request is submitted for the same web service, and the connection is reused.
Another possibility is that there are no more HTTP requests to the web service and the connection is left idle. (generate idle connection)

The latter case creates an idle connection. In addition, if too many connections are allocated to a certain network service, idle connections may also be generated. Because each newly submitted HTTP request will only commandeer an idle connection. Therefore, setting limits for idle connections is necessary in most cases and requires consideration.

3.2 Eliminate the generation of idle connections

If you want to completely eliminate the generation of idle connections, you can set the value of its DisableKeepAlives field to true during initialization. At this time, the value of the "Connection" header of the HTTP request will be set to "close". This tells the web service that this web connection does not have to be kept alive, and that it can be disconnected after the current HTTP transaction is complete.

In this way, every time an HTTP request is submitted, a new network connection will be generated. Doing so will significantly increase the load on the network service and the client. Therefore, under normal circumstances, we do not need to set the DisableKeepAlive field.

In net.Dialerthe type, there is also a field KeepAlive that looks similar. However, it is not a concept with the aforementioned HTTP persistent connection. KeepAlive acts directly on the underlying socket.

Behind KeepAlive is a liveness detection mechanism for network connections (more precisely, TCP connections). Its value is used to indicate how often probe packets are sent. When the value is not greater than 0, it means that this mechanism is not enabled. DefaultTransport will set the value of this field to 30 seconds.

Four,`http.Server`

http.ServerThe type http.Clientcorresponds to . http.ServerRepresents a server based on the HTTP protocol, or a network service.

4.1 `http.Server`Types of `ListenAndServe`methods

http.ServerThe function of the type ListenAndServemethod is to listen to a network address based on the TCP protocol and process the received HTTP request.

By default, this method will enable the liveness detection mechanism for network connections to ensure that the connection is persistent.
At the same time, this method will be executed until a serious error occurs or it is turned off by the outside world.

When turned off by the outside world, it returns an http.ErrServerClosederror value represented by the variable.

4.2 `ListenAndServe`What the method mainly does

func (srv *Server) ListenAndServe() error {
    
    
	if srv.shuttingDown() {
    
    
		return ErrServerClosed
	}
	addr := srv.Addr
	if addr == "" {
    
    
		addr = ":http"
	}
	ln, err := net.Listen("tcp", addr)
	if err != nil {
    
    
		return err
	}
	return srv.Serve(ln)
}

ListenAndServeThe method mainly does the following things:

Check the Addr field for the current http.Servertype's value.

The value of this field represents the network address that the current network service needs to use. That is: IP address and port number. If the value of this field is an empty string, ":http" is used instead.

That is, use any domain name and IP address that can represent this machine, and the port number is 80.
net.ListenStart the monitoring based on TCP protocol on the determined network address by calling the function.
Check net.Listenthe error value returned by the function.

If the error value is not nil, then return the value directly. Otherwise, prepare to accept and process incoming HTTP requests by calling the Serve method of the current value.

4.3 (Derivative question) `net.Listen`What does the function do

net.ListenWhat the function does:

Parse the IP address and port number implied by the network address contained in the parameter value;
According to a given network protocol, determine the monitoring method and start monitoring;

This can also be extended to the net.socket function and socket-related knowledge.

4.4 (derivative question) `http.Server`How does the type of Serve method accept and process HTTP requests

In a for loop, the Accept method of network monitoring will be called continuously,

	for {
    
    
		rw, err := l.Accept()
  }

This method returns two result values:

The first result value is net.Connof type, which represents the network connection that contains the new incoming HTTP request;
The second result value is errora type value representing a possible error.

If error is not nil, the loop is terminated unless it represents a transient error. If the error is transient, the next iteration of the loop will start executing some time later.

If the Accept method here does not return a non-nil error value, then the program here will wrap its first result value into a type of value , and then call the serve method of this type of value *http.connin the new goroutine . *http.connProcess the current HTTP request.

Related to HTTP requests, more derivative questions:

There are several states of this *http.conntype of value, which represent the stage of processing?
What readers and writers are used in the processing and what are their roles?
How does the program here call our custom processing function?

5. Thinking: How to gracefully stop the network service program based on HTTP protocol?

srv.Shutdown(context.Background()) The service can be stopped by means of RegisterOnShutdown, and the call when the service is stopped can be added through RegisterOnShutdown.