HTTP overview
An HTTP client must first make an HTTP request before it can receive a response from the HTTP server. The browser is the most common HTTP client. HTTP clients and HTTP servers are provided by different software developers, and they can be written in any programming language. HTTP strictly stipulates the data format of HTTP requests and HTTP responses. As long as the HTTP server and client programs both comply with HTTP, they can understand the messages sent by each other.
1. HTTP request format
The following is an example of an HTTP request
POST /hello.jsp HTTP/1.1
Accept:image/gif, image/jpeg, */*
Referer: http://localhost/login.htm
Accept-Language: en,zh-cn;q=0.5
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 10.0)
Host: localhost
Content-Length:43
Connection: Keep-Alive
Cache-Control: no-cache
username=root&password=12346&submit=submit
HTTP stipulates that an HTTP request consists of three parts, namely:
-
Request method, URI, HTTP version
- The first line of the HTTP request includes the request method, URI and protocol version, separated by spaces:
POST /hello.jsp HTTP/1.1
- The first line of the HTTP request includes the request method, URI and protocol version, separated by spaces:
-
Request Header
-
Request headers contain a lot of useful information about the client environment and the request body. For example, the request header can declare the type of browser, the language used, the type of request body, and the length of the request body, etc.
Accept:image/gif, image/jpeg, */* Referer: http://localhost/login.htm Accept-Language: en,zh-cn;q=0.5 //浏览器所用的语言 Content-Type: application/x-www-form-urlencoded //正文类型 Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 10.0) //浏览器类型 Host: localhost //远程主机 Content-Length:43 //正文长度 Connection: Keep-Alive Cache-Control: no-cache
-
-
Request Content
-
HTTP stipulates that the request header and the request body must be separated by a blank line (that is, a line with only CRLF symbols). This blank line is very important. It indicates that the request header has ended, followed by the request body. The request body can include the client's Form data submitted via POST
username=root&password=12346&submit=submit
-
2. HTTP response format
Below is an example of an HTTP response
HTTP/1.1 200 0K
Server: nio/1.1
Content-type: text/html; charset=GBK
Content-length:97
<html>
<head>
<title>helloapp</title>
</head>
<body >
<h1>hello</h1>
</body>
</htm1>
The HTTP response also consists of three parts, namely:
-
HTTP version, status code, description
- The first line of the HTTP response includes the version of HTTP used by the server, the status code, and a description of the status code. These three contents are separated by spaces.
-
Response Header
-
The response header also contains a lot of useful information like the request header, such as server type, body type and body length, etc.
Server: nio/1.1 //服务器类型 Content-type: text/html; charset=GBK //正文类型 Content-length:97 //正文长度
-
-
Response Content
-
The response body is the specific document returned by the server, most commonly an HTML web page. HTTP response headers and response body must also be separated by a blank line
<html> <head> <title>helloapp</title> </head> <body > <h1>hello</h1> </body> </htm1>
-
Create a blocking HTTP server
The following example (SimpleHttpServer) creates a very simple HTTP server that receives HTTP requests from client programs and prints them to the console. Then the HTTP request is simply parsed. If the client program requests access to login.htm, the web page will be returned. Otherwise, the hello.htm web page will be returned. The login.htm and hello.htm files are located in the root directory
SimpleHttpServer listens to port 80, works in blocking mode, and uses a thread pool to process each customer request.
public class SimpleHttpServer {
private int port = 80;
private ServerSocketChannel serverSocketChannel = null;
private ExecutorService executorService;
private static final int POOL MULTIPLE = 4;
private Charset charset = Charset.forName("GBK");
public SimpleHttpServer() throws IOException {
executorService= Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() * POOL MULTIPLE);
serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.socket().setReuseAddress(true);
serverSocketChannel.socket().bind(new InetSocketAddress(port));
System.out.println("服务器启动");
}
public void service() {
while (true) {
SocketChannel socketChannel = null;
try {
socketChannel = serverSocketChannel.accept();
executorService.execute(new Handler(socketChannel));
} catch (IOException e) {
e.printStackTrace();
}
}
}
public static void main(String args[])throws IOException {
new SimpleHttpServer().service();
}
public String decode(ByteBuffer buffer) {
......} //解码
public ByteBuffer encode(String str) {
......} //编码
//Handler是内部类,负责处理HTTP请求
class Handler implements Runnable {
private SocketChannel socketChannel;
public Handler(SocketChannel socketChannel) {
this.socketChannel = socketChannel;
}
public void run() {
handle(socketChannel);
}
public void handle(SocketChannel socketChannel) {
try {
Socket socket = socketChannel.socket();
ByteBuffer buffer = ByteBuffer.allocate(1024);
//接收HTTP请求,假定其长度不超过1024字节
socketChannel.read(buffer);
buffer.flip();
String request = decode(buffer);
//打印HTTP请求
System.out.print(request);
//生成HTTP响应结果
StringBuffer sb = new StringBuffer("HTTP/1.1 200 0K\r\n");
sb.append("Content-Type:text/html\r\n\r\n");
//发送HTTP响应的第1行和响应头
socketChannel.write(encode(sb.toString()));
FileInputStream in;
//获得HTTP请求的第1行
String firstLineOfRequest = request.substring(0, request.indexOf("\r\n"));
if(firstLineOfRequest.indexOf("login.htm") != -1) {
in = new FileInputStream("login.htm");
} else {
in = new FileInputStream("hello.htm");
}
FileChannel fileChannel = in.getChannel();
//发送响应正文
fileChannel.transferTo(0, fileChannel.size(), socketChannel);
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
if(socketChannel != null) {
//关闭连接
socketChannel.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
Create a non-blocking HTTP server
The following is a model of the non-blocking HTTP server example introduced in this section.
- HttpServer: the server main program, which starts the server
- AcceptHandler: Responsible for receiving client connections
- RequestHandler: Responsible for receiving the client's HTTP request, parsing it, then generating the corresponding HTTP response, and then sending it to the client.
- Request: represents HTTP request
- Response: represents HTTP response
- Content: Represents the body of the HTTP response
1. Server main program HttpServer
HttpServer only enables a single main thread, using non-blocking mode to receive client connections and send and receive data.
public class HttpServer {
private Selector selector = null;
private ServerSocketChannel serverSocketChannel = null;
private int port = 80;
private Charset charset = Charset.forName("GBK");
public HttpServer() throws IOException {
//创建Selector和ServerSocketChannel
//把ServerSocketchannel设置为非阻塞模式，绑定到80端口
......
}
public void service() throws IOException {
//注册接收连接就绪事件
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT, new AcceptHandler());
while(true) {
int n = selector.select();
if(n==0) continue;
Set readyKeys = selector.selectedKeys();
Iterator it = readyKeys.iterator();
while(it.hasNext()) {
SelectionKey key = null;
try {
key = (SelectionKey) it.next();
it.remove();
final Handler handler = (Handler) key.attachment();
handler.handle(key); //由 Handler 处理相关事件
} catch(IOException e) {
e.printStackTrace();
try {
if(key != null) {
key.cancel();
key.channel().close();
}
} catch(Exception ex) {
e.printStackTrace();
}
}
}
}
}
public static void main(String args[])throws Exception {
final HttpServer server = new HttpServer();
server.service();
}
}
2. ChannelIO class with auto-growing buffer
The custom ChannelIO class wraps SocketChannel and adds the function of automatically growing the buffer capacity. When calling the socketChannel.read(ByteBuffer bufer) method, if the buffer is full, even if there is unreceived data in the channel, the read method will not read any data, but will directly return 0, indicating that zero bytes have been read.
In order to read all the data in the channel, the buffer capacity must be large enough. There is a requestBuffer variable in the ChannelIO class, which is used to store the customer's HTTP request data. When the remaining capacity of the requestBuffer is less than 5% and there is still HTTP request data that has not been received, ChannelO will automatically expand the capacity of the requestBuffer. This function is controlled by resizeRequestBuffer () method completed
public class ChannelIO {
protected SocketChannel socketChannel;
protected ByteBuffer requestBuffer; //存放请求数据
private static int requestBufferSize = 4096;
public ChannelIO(SocketChannel socketChannel， boolean blocking) throws IOException {
this.socketChannel = socketChannel;
socketChannel.configureBlocking(blocking); //设置模式
requestBuffer = ByteBuffer.allocate(requestBufferSize);
}
public SocketChannel
() {
return socketChannel;
}
/**
* 如果原缓冲区的剩余容量不够，就创建一个新的缓冲区，容量为原来的两倍
* 并把原来缓冲区的数据拷贝到新缓冲区
*/
protected void resizeRequestBuffer(int remaining) {
if (requestBuffer.remaining() < remaining) {
ByteBuffer bb = ByteBuffer.allocate(requestBuffer.capacity() * 2);
requestBuffer.flip();
bb.put(requestBuffer); //把原来缓冲区中的数据拷贝到新的缓冲区
requestBuffer = bb;
}
}
/**
* 接收数据，把它们存放到requestBuffer
* 如果requestBuffer的剩余容量不足5%
* 就通过resizeRequestBuffer()方法扩充容量
*/
public int read() throws IOException {
resizeRequestBuffer(requestBufferSize/20);
return socketChannel.read(requestBuffer);
}
/** 返回requestBuffer，它存放了请求数据 */
public ByteBuffer getReadBuf() {
return requestBuffer;
}
/** 发送参数指定的 ByteBuffer 的数据 */
public int write(ByteBuffer src) throws IOException {
return socketChannel.write(src);
}
/** 把FileChannel的数据写到SocketChannel */
public long transferTo(FileChannel fc, long pos, long len) throws IOException {
return fc.transferTo(pos, len, socketChannel);
}
/** 关闭SocketChannel */
public void close() throws IOException {
socketChannel.close();
}
}
3. Handler interface responsible for handling various events
The Handler interface is responsible for handling various events. Its definition is as follows:
public interface Handler {
public void handle(SelectionKey key) throws IOException;
}
The Handler interface has two implementation classes: AcceptHandler and RequestHandler. AcceptHandler is responsible for handling receiving connection ready events, and RequestHandler is responsible for handling read ready and write ready events. More precisely, the RequestHandler is responsible for receiving the client's HTTP request and sending the HTTP response
4. The AcceptHandler class responsible for handling the reception of connection ready events
AcceptHandler is responsible for processing the receiving connection ready event, obtaining the SocketChannel connected to the client, and then registering the read ready event with the Selector, and creating a RequestHandler as an attachment to the SelectionKey. When the read ready event occurs, this RequestHandler will handle the event
public class AcceptHandler implements Handler {
public void handle(SelectionKey key) throws IOException {
ServerSocketChannel serverSocketChannel = (ServerSocketChannel) key.channel();
//在非阻塞模式下,serverSocketChannel.accept()有可能返回null
SocketChannel socketChannel = serverSocketChannel.accept();
if (socketChannel == null) return;
//ChannelIO设置为采用非阻塞模式
ChannelIO cio = new ChannelIO(socketChannel, false);
RequestHandler rh = new RequestHandler(cio);
//注册读就绪事件,把RequestHandler作为附件
socketChannel.register(key.selector(), SelectionKey.OP_READ, rh);
}
}
5. The RequestHandler class responsible for receiving HTTP requests and sending HTTP responses
The RequestHandler first receives the HTTP request through ChannelIO. After receiving all the data of the HTTP request, it parses the HTTP request data, creates the corresponding Request object, then creates the corresponding Response object based on the customer's request content, and finally sends the Response object. HTTP response data contained in . In order to simplify the program, RequestHandler only supports GET and HEAD request methods.
public class RequestHandler implements Handler {
private ChannelIO channelIO;
//存放HTTP请求的缓冲区
private ByteBuffer requestByteBuffer = null;
//表示是否已经接收到HTTP请求的所有数据
private boolean requestReceived = false;
//表示HTTP请求
private Request request = null;
//表示HTTP响应
private Response response = null;
RequestHandler(ChannelIO channelIO) {
this.channelIO = channelIO;
}
/** 接收HTTP请求,发送HTTP响应 */
public void handle(SelectionKey sk) throws IOException {
try {
//如果还没有接收HTTP请求的所有数据,就接收HTTP请求
if (request == null) {
if (!receive(sk)) return;
requestByteBuffer.flip();
//如果成功解析了HTTP请求,就创建一个Response对象
if (parse()) build();
try {
//准备HTTP响应的内容
response.prepare();
} catch (IOException x) {
response.release();
response = new Response(Response.Code.NOT_FOUND, new StringContent(x.getMessage()));
response.prepare();
}
if (send()) {
//如果HTTP响应没有发送完毕，则需要注册写就绪事件，以便在写就绪事件发生时继续发送数据
sk.interestOps(SelectionKey.OP_WRITE);
} else {
//如HTTP响应发送完毕，就断开底层连接，并且释放Response占用资源
channelIO.close();
response.release();
}
} else {
//如果已经接收到HTTP请求的所有数据
//如果HTTP响应发送完毕
if (!send()) {
channelIO.close();
response.release();
}
}
} catch (IOException e) {
e.printStackTrace();
channelIO.close();
if (response != null) {
response.release();
}
}
}
/**
* 接收HTTP请求,如果已经接收到了HTTP请求的所有数据,就返回true,否则返回false
*/
private boolean receive(SelectionKey sk) throws IOException {
ByteBuffer tmp = null;
//如果已经接收到HTTP请求的所有数据,就返回true
if (requestReceived) return true;
//如果已经读到通道的末尾,或者已经读到HTTP请求数据的末尾标志,就返回true
if ((channelIO.read() < 0) || Request.isComplete(channelIO.getReadBuf())) {
requestByteBuffer = channelIO.getReadBuf();
return (requestReceived = true);
}
return false;
}
/**
* 通过Request类的parse()方法,解析requestByteBuffer的HTTP请求数据
* 构造相应的Request对象
*/
private boolean parse() throws IOException {
try {
request = Request.parse(requestByteBuffer);
return true;
} catch (MalformedRequestException x) {
//如果HTTP请求的格式不正确,就发送错误信息
response = new Response(Response.Code.BAD_REQUEST, new StringContent(x))
}
return false;
}
/** 创建HTTP响应 */
private void build() throws IOException {
Request.Action action = request.action();
//仅仅支持GET和HEAD请求方式
if ((action != Request.Action.GET) && (action != Request.Action.HEAD)) {
response = new Response(Response.Code.METHOD_NOT_ALLOWED, new StringContent("Method Not Allowed"));
} else {
response = new Response(Response.Code.OK, new FileContent(request.uri()), action);
}
}
/** 发送HTTP响应，如果全部发送完毕，就返回false，否则返回true */
private boolean send() throws IOException {
return response.send(channelIO);
}
}
6. Request class representing HTTP request
When RequestHandler reads HTTP request data through ChannelIO, the data is placed in requestByteBuffer. When all the data of the HTTP request has been received, the data of requestByteBufer must be parsed, and then the corresponding Request object must be created. The Request object represents a specific HTTP request
public class Request {
//枚举类，表示HTTP请求方式
static enum Action {
GET,PUT,POST,HEAD;
}
public static Action parse(String s) {
if (s.equals("GET"))
return GET;
if (s.equals("PUT"))
return PUT;
if (s.equals("POST"))
return POST;
if (s,equals("HEAD"))
return HEAD;
throw new IllegalArgumentException(s);
}
private Action action; //请求方式
private String version; //HTTP版本
private URI uri; //URI
public Action action() {
return action; }
public String version() {
return version; }
public URI uri() {
return uri; }
private Request(Action a, String V, URI u) {
action = a;
version = v;
uri =u;
}
public String toString() {
return (action + " " + version + " " + uri);
}
private static Charset requestCharset = Charset.forName("GBK");
/**
* 判断ByteBuffer是否包含HTTP请求的所有数据
* HTTP请求以”r\n\r\n”结尾
*/
public static boolean isComplete(ByteBuffer bb) {
ByteBuffer temp = bb.asReadOnlyBuffer();
temp.flip();
String data = requestCharset.decode(temp).toString();
if(data.indexOf("r\n\r\n") != -1) {
return true;
}
return false;
}
/**
* 删除请求正文
*/
private static ByteBuffer deleteContent (ByteBuffer bb) {
ByteBuffer temp = bb.asReadOnlyBuffer();
String data = requestCharset.decode(temp).toString();
if(data.indexOf("\r\n\r\n") != -1) {
data = data.substrinq(0, data.indexOf("\r\n\r\n") + 4);
return requestCharset.encode(data);
}
return bb;
}
/**
* 设定用于解析HTTP请求的字符串匹配模式,对于以下形式的HTTP请求
* GET /dir/file HTTP/1.1
* Host: hostname
* 将被解析成:
* group[l] = "GET”
* group[2]="/dir/file"
* group[3]="1.1"
* group[4]="hostname"
*/
private static Pattern requestPattern =
Pattern.compile("\\A([A-Z]+) +([^]+) +HTTP/([0-9\\.]+)$"
+ ",*^Host:([]+)$.*\r\n\r\n\\z",
Pattern.MULTILINE | Pattern.DOTALL);
/** 解析HTTP请求，创建相应的Request对象 */
public static Request parse(ByteBuffer bb) throws MalformedRequestException {
bb = deleteContent(bb); //删除请求正文
CharBuffer cb = requestCharset.decode(bb); //解码
Matcher m = requestPattern.matcher(cb); //进行字符串匹配
//如果HTTP请求与指定的字符串式不匹配，说明请求数据不正确
if (!m.matches())
throw new MalformedRequestException();
Action a;
//获得请求方式
try {
a = Action.parse(m.group(1));
} catch (IllegalArgumentException x) {
throw new MalformedRequestException();
}
//获得URI
URI u;
try {
u=new URI("http://" + m.group(4) + m.group(2));
} catch (URISyntaxException x) {
throw new MalformedRequestException();
}
//创建一个Request对象，并将其返回
return new Request(a, m.group(3), u);
}
}
7. Response class representing HTTP response
The Response class represents the HTTP response. It has three member variables: code, headerBufer and content, which respectively represent the status code, response header and body in the HTTP response.
public class Response implements Sendable {
//枚举类，表示状态代码
static enum Code {
OK(200, "OK"),
BAD_REQUEST(400, "Bad Request"),
NOT_FOUND(404, "Not Found"),
METHOD_NOT_ALLOWED(405, "Method Not Allowed");
private int number;
private String reason;
private Code(int i, String r) {
number = i;
reason =r;
}
public String toString() {
return number + " " + reason;
}
}
private Code code; //状态代码
private Content content; //响应正文
private boolean headersOnly; //表示HTTP响应中是否仅包含响应头
private ByteBuffer headerBuffer = null; //响应头
public Response(Code rc, Content c) {
this(rc, c, null);
}
public Response(Code rc, Content c, Request.Action head) {
code = rc;
content = c;
headersOnly = (head == Request.Action.HEAD);
}
/** 创建响应头的内容，把它存放到ByteBuffer */
private ByteBuffer headers() {
CharBuffer cb = CharBuffer.allocate(1024);
while(true) {
try {
cb.put("HTTP/1.1").put(code.toString()).put(CRLF);
cb.put("Server: nio/1.1").put(CRLF);
cb.put("Content-type: ") .put(content.type()).put(CRIE);
cb.put("Content-length: ").put(Long.toString(content.length())).put(CRLF);
cb.put(CRLF);
break;
} catch (BufferOverflowException x) {
assert(cb.capacity() < (1 << 16));
cb = CharBuffer.allocate(cb.capacity() * 2);
continue;
}
}
cb.flip();
return responseCharset.encode(cb); //编码
}
/** 准备 HTTP 响应中的正文以及响应头的内容 */
public void prepare() throws IOException {
content.prepare();
headerBuffer= headers();
}
/** 发送HTTP响应，如果全部发送完毕，就返回false，否则返回true */
public boolean send(ChannelIO cio) throws IOException {
if (headerBuffer == null) {
throw new IllegalStateException();
}
//发送响应头
if (headerBuffer.hasRemaining()) {
if (cio.write(headerBuffer) <= 0)
return true;
}
//发送响应正文
if (!headersOnly) {
if (content.send(cio))
return true;
}
return false;
}
/** 释放响应正文占用的资源 */
public void release() throws IOException {
content.release();
}
}
8. Content interface representing the response body and its implementation class
The Response class has a member variable content, which represents the response body and is defined as the Content type.
public interface Content extends Sendable {
//正文的类型
String type();
//返回正文的长度
//在正文准备之前，即调用prepare()方法之前，length()方法返回“-1”
long length();
}
The Content interface inherits the Sendable interface, which represents the content that the server can send to the client.
public interface Sendable {
// 准备发送的内容
public void prepare() throws IOException;
// 利用通道发送部分内容，如果所有内容发送完毕，就返回false
//如果还有内容未发送，就返回true
//如果内容还没有准备好，就抛出 IlleqalstateException
public boolean send(ChannelIO cio) throws IOException;
//当服务器发送内容完毕，就调用此方法，释放内容占用的资源
public void release() throws IOException;
}
The Content interface has two implementation classes, StringContent and FileContent. StringContent represents the text in the form of a string, and FileContent represents the text in the form of a file.
The FileContent class has a member variable fleChannel, which represents the channel for reading files. The send() method of the FileContent class sends the data in the fileChannel to the SocketChannel of ChannelIO. If all the data in the file is sent, the send() method returns false.
public class FileContent implements Content {
//假定文件的根目录为"root"，该目录应该位于classpath下
private static File ROOT = new File("root");
private File file;
public FileContent(URI uri) {
file = new File(ROOT, uri.getPath().replace('/', File,separatorChar));
}
private String type = null;
/** 确定文件类型 */
public String type() {
if (type != null) return type;
String nm = file.getName();
if (nm.endsWith(".html") || nm.endsWith(".htm"))
type = "text/html; charset=iso-8859-1"; //HTML网页
else if ((nm.indexOf('.') < 0) || nm.endsWith(".txt"))
type = "text/plain; charset=iso-8859-1"; //文本文件
else
type = "application/octet-stream"; //应用程序
return type;
}
private FileChannel fileChannel = null;
private long length = -1; //文件长度
private long position = -1;//文件的当前位置
public long length() {
return length;
}
/** 创建 FileChannel 对象 */
public void prepare() throws IOException {
if (fileChannel == null)
fileChannel = new RandomAccessFile(file, "r").getChannel();
length = fileChannel.size();
position =0;
}
/** 发送正文，如果发送完毕，就返回 false，否则返回true */
public boolean send(ChannelIO channelIO) throws IOException {
if (fileChannel == null)
throw new IllegalStateException();
if (position < 0)
throw new IllegalStateException();
if (position >= length)
return false; //如果发送完毕，就返回false
position += channelIO,transferTo(fileChannel, position, length - position);
return (position < length);
}
public void release() throws IOException {
if (fileChannel != null) {
fileChannel.close(); //关闭fileChannel
fileChannel = null;
}
}
}