[Node hen]
Node.js in the stream (stream) - The Basics
1 ) What is a stream (stream)
Stream (stream) is the stream data abstract interfaces (abstract interface) in the Node.js. stream module provides basic API. Using these API can be easily implemented to construct the object stream interface.
Stream is readable, writable or read-write.
2 ) nodejs of several types of Stream
Node.js There are four basic types stream:
Readable - readable stream (fs.createReadStream ())
Writable - writable stream (fs.createWriteStream ())
Duplex - writable stream (net.Socket)
Transform - can be modified and transformed in the data flow during read and write Duplex (e.g. zlib.createDeflate ())
NodeJs operation on stream is encapsulated into a Stream module, this module is also referenced by a plurality of core modules.
const stream = require('stream');
In NodeJS processing of documents using the most flow to complete.
Trivial File
Device files (stdin, stdout)
Network File (http, net)
Note: In all NodeJs Stream (stream) are examples of EventEmitter
Example：
1. The contents of the file to read the stream data 1.txt
const FS = the require ( 'FS');
// create a readable stream (producers)
the let RS = fs.createReadStream ( './ 1.txt');
Provided by the fs module createReadStream () you can easily create a readable file stream. But we have to use Stream module directly, because fs inside the module has been cited Stream module and made a package. So that the flow (stream) is the stream data abstract interface in Node.js provided a basis to construct an object implemented Api stream interface.
var rs = fs.createReadStream(path,[options]);
1.path read the file path
2.options
- flags the file open operation, the default is 'r'
- mode permission bits 0o666
- The default encoding is null
- The index position start to start reading
- end index read end position (including the end position)
- highWaterMark read buffer default size 64kb
Node.js offers a variety of stream object. E.g:
- HTTP 请求 （request response）
- process.stdout they are all examples of flow.
2. Create a writable stream (consumer) readable stream processing
1.txt to write readable stream to 2.txt file then we need a writable stream
const FS = the require ( 'FS');
// Create a stream write
the let WS = fs.createWriteStream ( './ 2.txt');
// to flow into the readable writable stream file written by pipe so
rs .pipe (ws);
var ws = fs.createWriteStream(path,[options]);
1.path read the file path
2.options
- flags the file open operation, the default is 'w'
- mode permission bits 0o666
- The default encoding is utf8
- autoClose: true whether to automatically close the file
- highWaterMark read buffer default size 16kb
It is a method of pipe Readable stream corresponds to a "pipe", data must be from the upstream to the downstream pipe, i.e. the pipe from a writable readable stream flow.
The subsequent in-depth introduction pipe.
As shown above, we compared the document the tub filled with water, the water content of the document is, we use a tube (pipe) connecting the two buckets so that the water flows from one tub to another tub, so that realize slowly the transmission of large files.
3 ) Why should I use Stream
When users watch video online, suppose we returned to the user via HTTP request video content
const http = require('http');
const fs = require('fs');
http.createServer((req, res) => {
fs.readFile(videoPath, (err, data) => {
res.end(data);
});
}).listen(8080);
But this there are two obvious problems
1. Video files need to read all finished, to return to the user, so waiting a very long time
2. Video files into a full memory, too much memory
You can stream video files into memory a little bit, little by little, and then returned to the user, read part of the writing part. (Using the HTTP protocol Transfer-Encoding: chunked fragmented transmission characteristics), to optimize the user experience, while the cost of decreased memory
const http = require('http');
const fs = require('fs');
http.createServer((req, res) => {
fs.createReadStream(videoPath).pipe(res);
}).listen(8080);
4 ) readable stream (Readable Stream)
Readable stream (Readable streams) is an abstract of the source (source) to provide data.
E.g:
HTTP responses, on the client
HTTP requests, on the server
fs read streams
TCP sockets
process.stdin
All Readable implement an interface stream.Readable class definition.
Readable modes stream (Flowing and paused)
1. In the flowing mode, automatically reads data stream read from the bottom of the system, and supplies the data to the application via the event interface EventEmitter quickly.
2. In the paused mode, must explicitly call Stream.Read () method reads data segments from the stream.
All the initial operating mode is paused Readable stream can be switched by the following three ways to flowing modes:
Listener 'data' events
Call stream.resume () method
Call stream.pipe () method to send data to Writable
Flow pattern flowing
Mode is switched to the flow stream a data event listener
const rs = fs.createReadStream('./1.txt');
const ws = fs.createWriteStream('./2.txt');
rs.on('data', chunk => {
ws.write(chunk);
});
ws.on('end', () => {
ws.end();
});
If the write speed can not keep up the speed of reading, it may cause data loss. Normal circumstances should be, some finished, and then read the next paragraph, if not finished, then let read the stream to pause and wait to finish before continuing.
var FS = the require ( 'FS');
// read highWaterMark (3 bytes) of data, after filling cache read, and the data event trigger
var RS = fs.createReadStream (the sourcePath, {
HighWaterMark:. 3
});
var fs.createWriteStream = WS (destPath, {
HighWaterMark:. 3
});
rs.on ( 'data', function (the chunk) {// when data is flowing, the write data
iF (ws.write (the chunk) === false) {// If not finished, the stream reading pause
rs.pause ();
}
});
ws.on ( 'drain', function () {// buffer empty drain trigger event time to continue reading
rs.resume ();
});
rs.on ( 'End', function () {// when there is no data, streaming off
ws.end ();
});
Or a more direct pipe
FS .createReadStream ( the sourcePath ) .pipe ( FS .createWriteStream ( destPath ));
Pause mode paused
1. When there is no flow pipe (), call the pause () method can pause the stream
2.pipe (time), it is necessary to remove all data monitor the event, and then call unpipe () method
read(size)
Flow in pause mode, the program explicitly call read () method to get the data. read () method will be pulled from the internal buffer and returns the number of data, when no more data is available, returns null. read () does not trigger the 'data' event.
When using the read () method of reading data, if the size parameter is passed, then it will return the specified bytes of data; when the specified size are not available byte, null is returned. If you do not specify a size parameter, it will return all the data in the internal buffer.
NodeJS provides us with a readable, the event triggered when the stream is ready to read data, that is, to listen to this event, they notified us go read data like:
const FS = The require ( 'FS');
rs = fs.createReadStream (sourcePath);
// When you listen readable event will enter the pause mode
rs.on ( 'readable', () => {
Console .log ( rs._readableState.length);
// read parameter indicates if not read the entire data buffer
// read a field, if you want to find the stream read byte read buffer size in bytes or less, the process directly returns
let rs.read = CH (. 1);
});
Pause mode data buffer in the form of a linked list stored in the BufferList
5) can be written stream (Writable Stream)
Writable stream is an abstract data flow devices to consume data over the upstream flow, data can be written to the device through the flow of the program can be written, it is common local disk file or TCP, HTTP and other network response.
Writable examples include:
HTTP requests, on the client
HTTP responses, on the server
fs write streams
zlib streams
crypto streams
TCP sockets
child process stdin
process.stdout, process.stderr
All Writable streams implements the interface stream.Writable class definition.
process.stdin.pipe(process.stdout);
process.stdout is a stream write, to the standard output device-readable program stream process.stdin pass over the data writing. In understanding the basic understanding readable writable stream flow over very simple data flow is the direction in which the data stream is a source readable, writable stream is the destination, the link is bidirectional flow conduit.
Writable stream using
call flow example of a writable write () method can be written into the writable data stream
const FS = the require ( 'FS');
const RS = fs.createReadStream (the sourcePath);
const WS = fs.createWriteStream (destPath);
rs.setEncoding ( 'UTF-. 8'); // set the encoding format
rs.on ( 'data', the chunk => {
ws.write (the chunk); // write
});
Monitor the data stream will make the event readable readable flow into the flow mode, we call write writable stream () callback method in the incident, so that data is written to a writable stream abstract device destPath in.
write () method takes three parameters:
chunk {String | Buffer}, represents the data to be written
When the write data is the encoding time encoded string may be provided
Callback data is written after the callback
'Drain' event
If you call stream.write (chunk) method returns false, indicating that the current buffer is full, the flow at the appropriate time (after the cache cleared) trigger 'drain
const FS = the require ( 'FS');
const RS = fs.createReadStream (the sourcePath);
const WS = fs.createWriteStream (destPath);
rs.setEncoding ( 'UTF-. 8'); // set the encoding format
. RS ON ( 'data', the chunk => {
the let In Flag = ws.write (the chunk); // write data
IF (In Flag!) {// If buffer full suspension read
rs.pause ();
}
});
WS . ON ( 'Drain', () => {
rs.resume (); // continue to read the buffer is empty write
});
6 summary
Stream (stream) read stream into (flowing mode, and paused mode), the flow writable, readable and writable stream, provides a variety of Node.js stream object. For example, HTTP requests are just examples and process.stdout stream. stream module provides basic API. Using these API can be easily implemented to construct the object stream interface. They are called the underlying stream and the module package.
Reproduced in: https: //www.cnblogs.com/still1/p/11007940.html