Use Node.js server set up static resources

Transfer: https: //www.cnblogs.com/SheilaSun/p/7271883.html

For Node.js novice, to build a static resource server is a good exercise, from the simplest returns a file or wrong start, progressive enhancement, we can gradually deepen their understanding of the http. Then go to it, so that our hands are stained with network requests!

Note:

Of course, if used in the project express frame, with express.static line of code you can achieve the purpose of:
app.use(express.static('public'))
Here is what we want to achieve express.staticpart of the work behind, it is recommended to read the synchronization module source code.

basic skills

In no hurry to write the first line of code, but the first sort out what are the basic steps in terms of functionality.

Start a http server specified in the local port, waiting for requests from clients
When the request arrived, according to the request url, a static set of Base file directory, file location mapping to give
Check if the file exists
If the file does not exist, returns a status code 404, page sent to the client not found
If the file exists:
- Open the file to be read
- Set response header
- Send files to the client
Await the next request from the client

Basic functions

Code structure

Create a nodejs-static-webserverdirectory, run in the directory npm initinitialize a package.json file.

mkdir nodejs-static-webserver && cd "$_"
// initialize package.json
npm init

Then create the following directory:

-- config
---- default.json -- static-server.js -- app.js

default.json

{
    "port": 9527,
    "root": "/Users/sheila1227/Public",
    "indexPage": "index.html" }

default.jsStore some default configuration, such as port number, static files directory (root), the default page (indexPage) and so on. When such a request http://localhost:9527/myfiles/upon arrival. If According to roothave index.html in the directory after mapping obtained, according to our default configuration, will be sent back to the client content of index.html.

static-server.js

const http = require('http');
const path = require('path'); const config = require('./config/default'); class StaticServer { constructor() { this.port = config.port; this.root = config.root; this.indexPage = config.indexPage; } start() { http.createServer((req, res) => { const pathName = path.join(this.root, path.normalize(req.url)); res.writeHead(200); res.end(`Requeste path: ${pathName}`); }).listen(this.port, err => { if (err) { console.error(err); console.info('Failed to start server'); } else { console.info(`Server started on port ${this.port}`); } }); } } module.exports = StaticServer;

Within this module file, we declare a StaticServerclass, and to define its startmethod, in which the body creates a servertarget, monitor rquestevents, and bind to the server configuration file specified port. At this stage, we are for the time being does not make any request simply returns the requested file path to distinction. pathStandardized modules for connection and resolution path, so we do not specifically to deal with the differences between the operating systems.

app.js

const StaticServer = require('./static-server');

(new StaticServer()).start();

In this document, call the above static-servermodules, and create a StaticServer instance, calls its startmethod to start a static resource server. This latter document will not need to do other modifications, all perfect for static resources have occurred in the server static-server.jsinside.

In the directory start the program will see the successful launch of log:

> node app.js

Server started on port 9527

Access in the browser, server requests can see a direct return path.

Routing process

Before we have just returned to the client file location for any request it, and now we will replace it returns a true file:

    routeHandler(pathName, req, res) {
        
    }

    start() {
        http.createServer((req, res) => {
            const pathName = path.join(this.root, path.normalize(req.url));
            this.routeHandler(pathName, req, res); }).listen(this.port, err => { ... }); }

It will be routeHandlerto process the file sent.

Read static files

Before reading the document, with the fs.statpresence or absence of the detection file, if the file does not exist, the callback function receives an error response transmitted 404.

    respondNotFound(req, res) {
        res.writeHead(404, {
            'Content-Type': 'text/html'
        });
        res.end(`<h1>Not Found</h1><p>The requested URL ${req.url} was not found on this server.</p>`); } respondFile(pathName, req, res) { const readStream = fs.createReadStream(pathName); readStream.pipe(res); } routeHandler(pathName, req, res) { fs.stat(pathName, (err, stat) => { if (!err) { this.respondFile(pathName, req, res); } else { this.respondNotFound(req, res); } }); }

Note:

Read the file, here is streamed createReadStreaminstead readFile, because the latter will get the complete contents of the file before it is first read memory. Such a large case file, and then encounter multiple requests simultaneously access, readFileit does not come to bear. Using flow-readable file, the server need not wait until the data is fully loaded into memory and then sent back to the client, but while one side of the transmission block in response to a read. At this time in response to the first response will include the following:
Transfer-Encoding:chunked
By default, the end of the stream readable, writable flow end()method is called.

MIME Support

Now when the client returns the file, we did not specify the Content-Typehead, although you may find that access the text or picture viewer can show text or pictures correctly, but that does not conform to specifications . Any response containing an entity body (entity body) should indicate the file type in the head, otherwise no way of knowing when the browser type, will own guess (for possible extension from the file contents and the url). Response such as specifying the wrong type of display content can lead to confusion, such as the return is obviously a jpegpicture, but the error is specified header: 'Content-Type': 'text/html', will receive a pile of garbage.

Although there are ready-made mime modules are available, or are you here to achieve it, we tried to have a clearer understanding of the process.

Created in the root directory of mime.jsthe file:

const path = require('path');

const mimeTypes = {
    "css": "text/css", "gif": "image/gif", "html": "text/html", "ico": "image/x-icon", "jpeg": "image/jpeg", ... }; const lookup = (pathName) => { let ext = path.extname(pathName); ext = ext.split('.').pop(); return mimeTypes[ext] || mimeTypes['txt']; } module.exports = { lookup };

The module exposes a lookupmethod, returns the correct type in the path name, the type in ‘type/subtype’FIG. For an unknown type, as ordinary text processing.

Followed by static-server.jsintroducing the above mimemodule, in response to the returned document with the correct header field are:

    respondFile(pathName, req, res) {
        const readStream = fs.createReadStream(pathName);
        res.setHeader('Content-Type', mime.lookup(pathName));
        readStream.pipe(res);
    }

Re-run the program, you will see a picture can be displayed in a browser.

Note:

It should be noted that the Content-Typeexplanation should be the entity body of the original file type. Even after the entity content encoding (e.g. gzip, will be mentioned later), the field specifies the type of the entity body should remain before encoding.

Add other features

So far, several steps have been completed basic functions are listed, but there are still many areas for improvement, such as if the url user input corresponds to a directory on disk how to do? There, now for the same file (never changed) of repeated requests, the server is diligently over and over again sent back to the same file, these redundant data transmission, not only consumes bandwidth, but also to the server added burden. In addition, if the server can be compressed before sending content, but also help to reduce transmission time.

Read a file directory

At this stage, with the url: localhost:9527/testfolderto access a specified root folder exists under real testfolderfiles, and the server will complain:

Error: EISDIR: illegal operation on a directory, read

To add support for directory access, the step response of our refresh:

Request arrived, first determine whether there is a trailing slash url
If there is a trailing slash that user requests a directory
- If the directory exists
  - If there is a default page (such as index.html) directory, send a default page
  - If the default page content, send a directory listing does not exist
- If the directory does not exist, return 404
Without the trailing slash that user requests a file
- If the file exists, send files
- If the file does not exist, the same name directory to determine whether there is
  - If the directory exists, return to 301 and add the url in the original /as you want to go to a location
  - If the directory does not exist, return 404

We need to rewrite logic within routeHandler:

    routeHandler(pathName, req, res) {
        fs.stat(pathName, (err, stat) => {
            if (!err) {
                const requestedPath = url.parse(req.url).pathname;
                if (hasTrailingSlash(requestedPath) && stat.isDirectory()) {
                    this.respondDirectory(pathName, req, res); } else if (stat.isDirectory()) { this.respondRedirect(req, res); } else { this.respondFile(pathName, req, res); } } else { this.respondNotFound(req, res); } }); }

Continue to add respondRedirectmethods:

    respondRedirect(req, res) {
        const location = req.url + '/';
        res.writeHead(301, {
            'Location': location,
            'Content-Type': 'text/html' }); res.end(`Redirecting to <a href='${location}'>${location}</a>`); }

When the browser receives the response 301, based on the specified header locationfield value, issuing a new request to the server.

Continue to add respondDirectorymethods:

    respondDirectory(pathName, req, res) {
        const indexPagePath = path.join(pathName, this.indexPage);
        if (fs.existsSync(indexPagePath)) {
            this.respondFile(indexPagePath, req, res);
        } else { fs.readdir(pathName, (err, files) => { if (err) { res.writeHead(500); return res.end(err); } const requestPath = url.parse(req.url).pathname; let content = `<h1>Index of ${requestPath}</h1>`; files.forEach(file => { let itemLink = path.join(requestPath,file); const stat = fs.statSync(path.join(pathName, file)); if (stat && stat.isDirectory()) { itemLink = path.join(itemLink, '/'); } content += `<p><a href='${itemLink}'>${file}</a></p>`; }); res.writeHead(200, { 'Content-Type': 'text/html' }); res.end(content); }); } }

When you need to return a directory listing, through all the content and create a link for each, as part of the return of the document. Note that, for subdirectories href, add a trailing slash extra, to avoid another redirect when accessing a subdirectory.

Test it in a browser, enter the localhost:9527/testfolderspecified rootdirectory is not named testfolderfile, but there is the same name directory, and therefore for the first time will receive a redirect response, and initiate a new request to the directory.

Cache Support

In order to reduce data transmission, reducing the number of requests, continue to add caching support. First comb caching process flow:

If this is your first visit, request packet header does not contain relevant fields, the server performs the following before sending the file:
- Such as server support ETag, set up ETaghead
- Such as server support Last-Modified, set up Last-Modifiedhead
- Set Expireshead
- Setting Cache-Controlhead (set its max-agevalue)
These markers will be able to save the time to bring the browser receives the response, and the next request and ETagcorresponding request header If-None-Matchor a Last-Modifiedheader corresponding to the request If-Modified-Since.
If a duplicate request:
- Determine whether the browser cache expiration (through Cache-Controland Expiresdetermine)
  - If not expired, the direct use of cache content, which is strong cache hit, and no new request
  - If expired, it will initiate a new request, and the request will be put on If-None-Match, or If-Modified-Since, both, or both
  - Server receives the request, the cache re-validation freshness:
    - First check whether there is a request If-None-Matchheaders, not then continue to the next step, there will be the latest ETag matches the value of the document, it is considered a failure cache is not new, successful, continue to the next
    - Then check whether there is a request If-Modified-Sinceheader, no step is retained on the verification result, there will be the value of the document was last modified comparative validation failure is considered stale cache, the cache is considered successful fresh
    When neither the presence of two or header verification result is not fresh, and transmits the latest file 200, and updates the freshness header.
    
    When the verification result is still fresh cache (that is, weak cache hit), no need to send files, send only 304, and updates the freshness in the header

In order to enable or disable one of the authentication mechanism, we add the following configuration items in the configuration file:

default.json：

{
    ...
    "cacheControl": true,  "expires": true,  "etag": true,  "lastModified": true,  "maxAge": 5 }

Here order to be able to test the cache expires, the expiration time set into a very small 5 seconds.

In StaticServerreceiving the configuration classes:

class StaticServer {
    constructor() {
         ...
        this.enableCacheControl = config.cacheControl; this.enableExpires = config.expires; this.enableETag = config.etag; this.enableLastModified = config.lastModified; this.maxAge = config.maxAge; }

Now we're going in the original respondFileTransCanada a bar before, the increase is to return to the logic 304 or 200.

    respond(pathName, req, res) {
        fs.stat(pathName, (err, stat) => {
            if (err) return respondError(err, res);
            this.setFreshHeaders(stat, res); if (this.isFresh(req.headers, res._headers)) { this.responseNotModified(res); } else { this.responseFile(pathName, res); } }); }

Before preparing to return to the file, depending on the configuration, add the cache-related headers of the response.

    generateETag(stat) {
        const mtime = stat.mtime.getTime().toString(16);
        const size = stat.size.toString(16);
        return `W/"${size}-${mtime}"`; } setFreshHeaders(stat, res) { const lastModified = stat.mtime.toUTCString(); if (this.enableExpires) { const expireTime = (new Date(Date.now() + this.maxAge * 1000)).toUTCString(); res.setHeader('Expires', expireTime); } if (this.enableCacheControl) { res.setHeader('Cache-Control', `public, max-age=${this.maxAge}`); } if (this.enableLastModified) { res.setHeader('Last-Modified', lastModified); } if (this.enableETag) { res.setHeader('ETag', this.generateETag(stat)); } }

It should be noted that the above uses a ETagweak validator, cache files and does not guarantee that the file on the server is exactly the same. On how to implement strong authentication device, you can refer etag source packages.

Here is how to determine whether the cache is still fresh:

    isFresh(reqHeaders, resHeaders) {
        const  noneMatch = reqHeaders['if-none-match'];
        const  lastModified = reqHeaders['if-modified-since'];
        if (!(noneMatch || lastModified)) return false; if(noneMatch && (noneMatch !== resHeaders['etag'])) return false; if(lastModified && lastModified !== resHeaders['last-modified']) return false; return true; }

Note that, http header field names are case-insensitive (but http method should be capitalized), so usually in the browser will see the upper or lower header field.

But nodethe httpmodule header fields have turned into lowercase, so use them in your code is more convenient. So visit header use lowercase, such as reqHeaders['if-none-match']. However, you can still use req.rawreq.rawHeadersto access the original headers, it is a [name1, value1, name2, value2, ...]form of array.

Now to test, because the cache valid time setting is minimal 5s, it is hardly strong cache hit, so a second visit to file a new request, because the server did not do anything to change the file, it will return 304.

304

Now look at this picture to modify the request, such as modify size, purpose is to re-verify the server failure, and therefore must send 200 new files to the client.

200

The next big change to the effective time of the cache some, such as 10 minutes, then repeat the request within 10 minutes, will hit strong cache, the browser will not initiate a new request to the server (network but was still able to observe this request ).

cache

Content-Encoding

Server before sending large documents, compress them, you can save time transmission. The process is:

Browser when you visit the site, the default will carry Accept-Encodinghead
Server receives the request, if the found Accept-Encodingrequest header, and supports the file type of compression, compression entity body response (no compression head), along with Content-Encodingthe first portion
The browser receives the response, if found to have Content-Encodingthe head, according to the value specified format decompression messages

For pictures of these already highly compressed files without additional compression again. Therefore, we need to configure a field, which indicates the need for compressed file types for.

default.json

{
    ...
    "zipMatch": "^\\.(css|js|html)$"
}

static-server.js

    constructor() {
        ...
        this.zipMatch = new RegExp(config.zipMatch);
    }

A zlibmodule to achieve compression stream:

    compressHandler(readStream, req, res) {
       const acceptEncoding = req.headers['accept-encoding'];
       if (!acceptEncoding || !acceptEncoding.match(/\b(gzip|deflate)\b/)) {
           return readStream; } else if (acceptEncoding.match(/\bgzip\b/)) { res.setHeader('Content-Encoding', 'gzip'); return readStream.pipe(zlib.createGzip()); } else if (acceptEncoding.match(/\bdeflate\b/)) { res.setHeader('Content-Encoding', 'deflate'); return readStream.pipe(zlib.createDeflate()); } }

Because the configuration of the images without compression, in a browser test will find no response to the requested image Content-Encodinghead.

Range request

The final step, the server support range requests, allowing the client to request only part of the document. Its process is:

The client sends a request to the server
The server response, attach Accept-Rangesthe head (represented by the range value represented by the unit, is usually "bytes"), which tells the client request acceptance range
The client sends a new request, accompanied by Rangesthe head, telling the server request is a range
Range server receives the request, in response Points:
- Effective range, returned from the server 206 Partial Content, transmitting the contents within the specified range, and Content-Rangethe specified range of the head
- Disabling range, returned from the server 416 Requested Range Not Satisfiable, and Content-Rangethe acceptable range specified

Request Rangesheader format (here, the request is not considered a multi-range):

Ranges: bytes=[start]-[end]

Which start and end at the same time must not have:

If end is omitted, the server should return all bytes from the start position after the start
If omitted, start, end value refers to the last server returns the number of bytes
If none is omitted, the server returns the bytes between the start and end

Response Content-Rangeheader in two formats:

When the range of the effective return 206:
```
Content-Range: bytes (start)-(end)/(total)
```
When the range is invalid return 416:
```
Content-Range: bytes */(total)
```

Add Function request processing range:

    rangeHandler(pathName, rangeText, totalSize, res) {
        const range = this.getRange(rangeText, totalSize);
        if (range.start > totalSize || range.end > totalSize || range.start > range.end) { res.statusCode = 416; res.setHeader('Content-Range', `bytes */${totalSize}`); res.end(); return null; } else { res.statusCode = 206; res.setHeader('Content-Range', `bytes ${range.start}-${range.end}/${totalSize}`); return fs.createReadStream(pathName, { start: range.start, end: range.end }); } }

By Postman to test. In the specified rootcreate a test file in the folder:

testfile.js

This is a test sentence.

Request returns the first six bytes "This" return 206:

206

Request a return invalid range 416:

416

Read command line arguments

So far, we have completed the basic functions of the static server. But every time you need to modify the configuration, you must modify the default.jsonfile, very inconvenient, if we can accept the command line arguments like, you can help yargs module to complete.

var options = require( "yargs" )
    .option( "p", { alias: "port",  describe: "Port number", type: "number" } ) .option( "r", { alias: "root", describe: "Static resource directory", type: "string" } ) .option( "i", { alias: "index", describe: "Default page", type: "string" } ) .option( "c", { alias: "cachecontrol", default: true, describe: "Use Cache-Control", type: "boolean" } ) .option( "e", { alias: "expires", default: true, describe: "Use Expires", type: "boolean" } ) .option( "t", { alias: "etag", default: true, describe: "Use ETag", type: "boolean" } ) .option( "l", { alias: "lastmodified", default: true, describe: "Use Last-Modified", type: "boolean" } ) .option( "m", { alias: "maxage", describe: "Time a file should be cached for", type: "number" } ) .help() .alias( "?", "help" ) .argv;

Chou Chou Han help command output:

help command

So that you can transfer port, default page and so on at the command line:

node app.js -p 8888 -i main.html

reference

Source

Poke my repo GitHub: nodejs-static-webserver

Bowen also simultaneously in GitHub, welcome to discuss and correct me: the use of static resources to build Node.js server