This article mainly introduces the development and application of Netty based on the HTTP protocol
1. Concept introduction
The HTTP protocol is an object-oriented protocol at the application layer built on top of the TCP transport protocol.
The main features are as follows:
1. Support Client/Server mode;
2. Simple. When a client requests a service from the server, it only needs to specify the service URL and carry the necessary request parameters or message body;
3. Flexible. HTTP allows the transmission of data objects of any type, and the content type of the transmission is marked by the Content-Type in the HTTP message header;
4. Stateless. Refers to the protocol has no memory capability for transaction processing.
The format of the HTTP URL is as follows:
http://host[":"port][abs_path]
http means to locate network resources through the HTTP protocol; host means a legitimate Internet host domain name or IP address; port specifies a port number, if it is empty, the default port 80 is used; abs_path specifies the URI of the requested resource.
HTTP request message HttpRequest:
An HTTP request consists of three parts: the request line, the message header, and the request body.
Request line format: Method Request-URI HTTP-Version CRLF
Method represents the request method, Request-URI represents the Uniform Resource Identifier, HTTP-Version represents the HTTP protocol version of the request, and CRLF represents carriage return and line feed.
Request method:
Message header:
HTTP response message HttpResponse
After processing the request of the HTTP client, the HTTP server returns a response message to the client. The HTTP response is also composed of three parts: the status line, the message header, and the response body.
Status line format: HTTP-Version Status-Code Reason-Phrase CRLF
Status-Code indicates the response status code returned by the server, and Reason-Phrase indicates the error message returned by the server.
The status code consists of three numbers, with the following 5 possible values.
(1) 1xx: Indication information. Indicates that the request has been received and processing continues.
(2) 2xx: Success. Indicates that the request has been successfully received, understood, accepted.
(3) 3xx: Redirect. Further action must be taken to complete the request.
(4) 4xx: Client error. The request has a syntax error or the request cannot be fulfilled.
(5) 5xx: Server error. The server failed to process the request.
Common response headers:
Location: The response header field is used to redirect the receiver to a new location. The Location response header field is often used when changing the domain name.
Server: The response header field contains the software information used by the server to process the request, which corresponds to the User-Agent request header field.
WWW-Authenticate: Must be included in the 401 Unauthorized response message. When the client receives the 401 response message and sends the Authorization header field to request the server to verify it, the server response header contains this header field.
Second, the example code
Since Netty is an asynchronous event-driven framework by nature, the HTTP protocol stack developed based on the NIO TCP protocol stack is also asynchronous and non-blocking.
Netty's HTTP protocol stack has excellent performance and reliability, and is very suitable for applications in non-Web container scenarios.
Example scenarios are as follows:
The file server uses the HTTP protocol to provide external services. When the client accesses the file server through a browser, the access path is checked. If the check fails, an HTTP 403 error is returned, and the page cannot be accessed. Directory of files, each directory or file is a hyperlink that can be accessed recursively.
HTTP server development:
package com.huawei.netty.http;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.codec.http.HttpRequestDecoder;
import io.netty.handler.codec.http.HttpResponseEncoder;
import io.netty.handler.stream.ChunkedWriteHandler;
/**
* Created by liuzhengqiu on 2017/11/4.
*/
public class HttpFileServer
{
private static final String DEAFAULT_URL = "/src/main/java/com/huawei/netty";
public void run(final int port,final String url) throws Exception
{
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try
{
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup,workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel socketChannel) throws Exception
{
socketChannel.pipeline()
.addLast("http-decoder",new HttpRequestDecoder())
.addLast("http-aggregator",new HttpObjectAggregator(65536))
.addLast("http-encoder",new HttpResponseEncoder())
.addLast("http-chunked",new ChunkedWriteHandler())
.addLast("fileServerHanlder",new HttpFileServerHandler(url));
}
});
ChannelFuture future = bootstrap.bind("127.0.0.1",port).sync();
System.out.println("HTTP file directory server starts, the URL is: http://127.0.0.1:"+port+url);
future.channel().closeFuture().sync();
}
finally
{
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
public static void main(String[] args) throws Exception {
new HttpFileServer().run(8080,DEAFAULT_URL);
}
}
Added a decoder for HTTP request messages to ChannelPipeline, and then added a HttpObjectAggregator decoder, which is used to convert multiple messages into a single FullHttpRequest or FullHttpResponse, because the HTTP decoder generates multiple messages in each HTTP message. message object.
The main function of ChunkedHandler is to support asynchronous sending of large code streams without occupying too much memory and preventing Java memory overflow errors.
package com.huawei.netty.http;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.*;
import io.netty.handler.codec.http.*;
import io.netty.handler.stream.ChunkedFile;
import io.netty.util.CharsetUtil;
import javax.activation.MimetypesFileTypeMap;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.RandomAccessFile;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.util.regex.Pattern;
import static io.netty.handler.codec.http.HttpHeaders.Names.CONNECTION;
import static io.netty.handler.codec.http.HttpHeaders.Names.CONTENT_TYPE;
import static io.netty.handler.codec.http.HttpHeaders.Names.LOCATION;
import static io.netty.handler.codec.http.HttpHeaders.isKeepAlive;
import static io.netty.handler.codec.http.HttpHeaders.setContentLength;
import static io.netty.handler.codec.http.HttpResponseStatus.*;
/**
* Created by liuzhengqiu on 2017/11/4.
*/
public class HttpFileServerHandler extends SimpleChannelInboundHandler<FullHttpRequest>
{
private final String url;
private static final Pattern INSECURE_URI = Pattern.compile(".*[<>&\"].*");
private String sanitizeUri(String uri)
{
try
{
uri = URLDecoder.decode(uri,"UTF-8");
}
catch (UnsupportedEncodingException e)
{
throw new Error();
}
if (!uri.startsWith(url))
{
return null;
}
if (!uri.startsWith("/"))
{
return null;
}
uri = uri.replace('/', File.separatorChar);
if (uri.contains(File.separator + '.')
|| uri.contains('.'+File.separator)||uri.startsWith(".")
|| uri.endsWith(".")
|| INSECURE_URI.matcher(uri).matches())
{
return null;
}
return System.getProperty("user.dir") + File.separator + uri;
}
public HttpFileServerHandler(String url)
{
this.url = url;
}
private static void sendError(ChannelHandlerContext ctx, HttpResponseStatus status)
{
FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1,status,
Unpooled.copiedBuffer("Failure:"+status.toString()+"\r\n", CharsetUtil.UTF_8));
response.headers().set(CONTENT_TYPE,"text/plain;charset=UTF-8");
ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
}
@Override
protected void channelRead0(ChannelHandlerContext channelHandlerContext, FullHttpRequest fullHttpRequest) throws Exception
{
if (!fullHttpRequest.getDecoderResult().isSuccess())
{
sendError(channelHandlerContext,BAD_REQUEST);
return ;
}
if (fullHttpRequest.getMethod() != HttpMethod.GET)
{
sendError(channelHandlerContext,METHOD_NOT_ALLOWED);
return ;
}
final String uri = fullHttpRequest.getUri();
final String path = sanitizeUri(uri);
if (path == null)
{
sendError(channelHandlerContext,FORBIDDEN);
return ;
}
File file = new File(path);
if (file.isHidden() || !file.exists())
{
sendError(channelHandlerContext,NOT_FOUND);
return;
}
if (file.isDirectory())
{
if(uri.endsWith("/"))
{
sendListing(channelHandlerContext,file);
}
else
{
sendRedirect(channelHandlerContext,uri + '/');
}
return;
}
if (!file.isFile())
{
sendError(channelHandlerContext,FORBIDDEN);
return;
}
RandomAccessFile randomAccessFile = null;
try
{
randomAccessFile = new RandomAccessFile(file,"r");
}catch (FileNotFoundException fnfe)
{
sendError(channelHandlerContext,NOT_FOUND);
return;
}
long fileLength = randomAccessFile.length();
HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1,OK);
setContentLength(response,fileLength);
setContentTypeHeader(response,file);
if (isKeepAlive(fullHttpRequest))
{
response.headers().set(CONNECTION,HttpHeaders.Values.KEEP_ALIVE);
}
channelHandlerContext.write(response);
ChannelFuture sendFileFuture;
sendFileFuture = channelHandlerContext.write(new ChunkedFile(randomAccessFile,0,fileLength,8192),
channelHandlerContext.newProgressivePromise());
sendFileFuture.addListener(new ChannelProgressiveFutureListener() {
@Override
public void operationProgressed(ChannelProgressiveFuture channelProgressiveFuture, long progress, long total) throws Exception
{
if (total < 0)
{
System.err.println("Transfer progress: "+progress);
}
else
{
System.err.println("Transfer progress:"+progress+"/"+total);
}
}
@Override
public void operationComplete(ChannelProgressiveFuture channelProgressiveFuture) throws Exception
{
System.out.println("Transfer complete.");
}
});
ChannelFuture lastContentFuture = channelHandlerContext.writeAndFlush(LastHttpContent.EMPTY_LAST_CONTENT);
if (!isKeepAlive(fullHttpRequest))
{
lastContentFuture.addListener(ChannelFutureListener.CLOSE);
}
}
private static void setContentTypeHeader(HttpResponse response,File file)
{
MimetypesFileTypeMap mimetypesFileTypeMap = new MimetypesFileTypeMap();
response.headers().set(CONTENT_TYPE,mimetypesFileTypeMap.getContentType(file.getPath()));
}
private static final Pattern ALLOW_FILE_NAME = Pattern.compile("[A-Za-z0-9][-_A-Za-z0-9\\.]*");
private static void sendListing(ChannelHandlerContext ctx,File dir)
{
FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1,OK);
response.headers().set(CONTENT_TYPE,"text/html;charset=UTF-8");
StringBuilder buf = new StringBuilder();
String dirPath = dir.getPath();
buf.append("<!DOCTYPE html>\r\n");
buf.append("<html><head><title>");
buf.append(dirPath);
buf.append("目录：");
buf.append("</title></head><body>\r\n");
buf.append("<h3>");
buf.append(dirPath).append(" 目录： ");
buf.append("</h3>\r\n");
buf.append("<ul>");
buf.append("<li>链接:<a href=\"../\">..</a></li>\r\n");
for (File file : dir.listFiles())
{
if (file.isHidden() || !file.canRead())
{
continue;
}
String name = file.getName();
if (!ALLOW_FILE_NAME.matcher(name).matches())
{
continue;
}
buf.append("<li>链接:<a href=\"");
buf.append(name);
buf.append("\">");
buf.append(name);
buf.append("</a></li>\r\n");
}
buf.append("</ul></body></html>\r\n");
ByteBuf buffer = Unpooled.copiedBuffer(buf,CharsetUtil.UTF_8);
response.content().writeBytes(buffer);
buffer.release();
ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
}
private static void sendRedirect(ChannelHandlerContext ctx,String newUri)
{
FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1,FOUND);
response.headers().set(LOCATION,newUri);
ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
}
}
Start the HTTP file server and the results are as follows:
3. Summary
It mainly introduces the entry-level application of the Netty HTTP protocol stack. Currently the most popular is the form of HTTP+XML development or Restful+JSON. It will be further introduced later