"C++11/14 Advanced Programming: Exploring the Boost Library" Notes
The boost.iostreams library is built on the IO stream framework of the standard library. It defines new stream processing concepts such as device, source, sink, and filter, and constructs a new set of easy-to-use and customized stream processing frameworks that can be used almost anywhere. Stream to process any data, such as string processing, base64 encoding and decoding, compression and decompression, encryption and decryption, etc.
Getting Started Example
Simple example
Use standard stream-like functionality:
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/array.hpp> //预定义的数组设备头文件
namespace io = boost::iostreams;
using namespace io;
int main()
{
char str[] = "123";
array_source asrc(str,str+3); //一个数组源设备，注意构造函数
stream<array_source> in(asrc); //搭配源设备定义输入流
char c1,c2,c3;
in >> c1 >> c2; //使用读取操作从流中读取数据
assert(c1 == '1' && c2 == '2');
in.get(c3); //可以调用标准流的成员函数
assert(c3 == '3' && in.good()); //查看流的状态，正常
assert(in.get(c3).eof()); //流已经结束
char str2[10];
array_sink asnk(str2); //定义接收设备，支持直接传入数组
stream<array_sink> out(asnk); //搭配接收设备定义输出流
out << 'a' << 'b' << 'c'; //使用写入操作符向流写入数据
assert(str2[0] == 'a' && str2[2] == 'c');
return 0;
}Three components of the iostreams library are used here: array_source, array_sink and stream, and their usage is similar to cin/cout.
array_source is a device provided by the iostreams library. It can adapt a character buffer to a source device. With the source device, the stream can be connected to this device using the template parameter + constructor method to form a stream that is fully compatible with the standard stream. new input stream. The device connected to the stream is the source device. The source device is readable but not writable, so the new stream must be an input (read-only) stream.
array_sink is another device provided by the iostreams library. It can adapt a character buffer to a receiving device. After the stream is connected to the array_sink, it becomes an output stream and data can be written. The writing operations to the stream will eventually be received by the character buffer. .
Complex example
Demonstrations of the most important io::copy() algorithm in filter streams, pipes, and iostreams libraries have been added to more complex examples:
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/array.hpp> //预定义的数组设备头文件
#include <boost/iostreams/filtering_stream.hpp> //过滤流头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/filter/counter.hpp> //计数过滤器头文件
namespace io = boost::iostreams;
using namespace io;
int main()
{
char arr[] = "12345678";
stream<array_source> in(arr,arr+8); //直接创建输入流
std::string str;
filtering_ostream out( //输出过滤流，需连接sink设备作为链结束
counter() | //预定义的计数过滤器
io::back_inserter(str)); //适配标准容器作为接收设备
io::copy(in,out);
assert(str.size() == 8);
assert(str == arr);
assert(out.component<counter>(0)->characters() == 8);//获取计数结果
return 0;
}The core function of this example is to use filtering streams. filtering_ostream declares a filtering stream out for output (writable). The difference from the basic stream is that it can not only connect receiving devices, but also filters, and can also connect these devices. Connected into a processing chain. Two device objects are passed in the constructor of the filter stream out, and the overloaded operator | is used to connect them. " | " is called the pipe operator, which is similar to the usage of the pipe operator in UNIX. Data can flow from one device to another device through a pipe.
The first device in the filter stream device chain is counter, which is a filter predefined by the iostreams library and can count the number of characters and lines that pass the filter. The second device uses the io::back_inserter() function, which is a receiving device generator that can adapt a standard container into a receiving device.
Finally, the io::copy() algorithm is called to copy the data from an input stream to the output stream. In the entire process, the data starts from the input stream, first flows through the counter filter, and then flows into the adapted string container to complete the flow. Processing.
The last line calls the member function component() of the filter stream, returns the nth device in the device chain of the stream, here is the first filter counter, and then calls the member function characters() of counter to obtain the statistical result of the number of characters.
Device characteristics
The most basic characteristic of a device is the "char type" they can process. The minimum requirement for all devices in a stream to work together is that they must process the same "char type". The "character type" of the device can be obtained using the meta-function char_type_of<T>::type, similar to the standard library's std::char_traits<Ch>::char_type.
Another important feature of a device is its mode. In the iostreams library, mode refers to the input and output (read and write) access characteristics of the device, which is closer to the file mode of C language or the traversal concept of new iterators. There are four most commonly used modes: input mode, output mode, bidirectional mode (can perform read and write operations on two unrelated character sequences, such as std::iostream), positionable mode (can perform on one character sequence) Perform read and write operations, but the location of the operations can be relocated, such as std::fstream). Use mode_of<T> to get the mode of the device.
equipment
Device is the most basic concept in the iostreams library. It is located in the header file <boost/iostreams/concepts.hpp>. It is an object that can perform read or write operations on the sequence. Its behavior depends on its mode. To determine whether a class is a device, you can use the meta-function is_device<T>.
Device is a template class, a conceptual class only used for inheritance. It has two template parameters. The parameter Mode determines the mode of the device, and the parameter Ch is the device. The character type that can be processed, the default is narrow character char.
Device also has two very important internal type definitions, namely char_type and category, which mark the character type and classification information of the device respectively, and can be obtained using the corresponding meta-functions char_type_of and category_of.
device also defines several specializations, which represent some sub-concepts of the device, such as:
typedef device<input> source; //char源设备
typedef device<output> sink; //char接收设备iostreams provides several predefined devices: array devices, standard container devices, file devices, empty devices, etc.
array device
The array device is located in the header file <boost/iostreams/device/array.hpp>, which provides access to memory character sequences and can adapt the character buffer to a seekable (locatable mode) device. The array device is not responsible for memory management of the buffer, so it is usually used with a static array or std::vector.
There are three array devices: array_source (source device), array_sink (receiving device) and array (locatable device). These three array devices are actually char type specializations of basic_array_source, basic_array_sink and basic_array. They only differ in mode. The final implementation is the same class array_adapter.
template<typename Mode,typename Ch>
class array_adapter
{
public:
typedef Ch char_type; //字符类型
typedef std::pair<char_type*,char_type*> pair_type;
struct category
:public Mode,public device_tag,public direct_tag
{ };
array_adapter(char_type* begin,char_type* end); //构造函数
array_adapter(char_type* begin,std::size_t length);
array_adapter(char_type (&ar)[N])
:begin_(ar),end_(ar + N){ }
//返回pair对象，标记数组设备所能控制的序列范围，即成员变量begin_和end_
pair_type input_sequence();
pair_type output_sequence();
private:
char_type* begin_; //数组位置指针
char_type* end_;
}Usage: array_source specializes the char type. If you want to change and customize other data types, you cannot directly use array_source, array_sink and other devices that process char. Instead, use template classes such as basic_array_source to specialize yourself, such as:
int buf1[10] = {
1,2,3},buf2[10];
basic_array_source<int> ai(buf1); //使用int类型的源设备
basic_array<int> ar(buf2); //使用int类型的可定位设备
stream<basic_array_source<int> > in(ai);
stream<basic_array<int> > out(ar);
io::copy(in,out);Standard container equipment
If you want to adapt a standard container to a source device, you need to use the function boost::make_iterator_range() provided by the boost.range library. You can pass the container directly to the filter stream to create a stream that can be used for input. Example:
string str("123");
filtering_istream in(make_iterator_range(str));
vector<char> v(str.begin(),str.end());
filtering_istream in2(make_iterator_range(v));If you want to adapt the standard container to a receiving device, the iostreams library provides an adaptation class back_insert_device, located in the header file <boost/iostreams/device/back_inserter.hpp>, which calls the insert() operation of the standard container to the end of the container. Append data, much like std::back_insert_iterator. For ease of use, the iostreams library provides the factory function back_inserter(Container& cnt), which can directly return an adapted receiving device:
string str("123");
std::vector<char> v;
io::copy(make_iterator_range(str),io::back_inserter(v));The above two methods can use standard containers for stream processing, but in fact there is no container device that conforms to the device concept. The iostreams library provides three available template classes in the sample code <libs/iostreams/example/container_device.hpp>: container_source, container_sink and container_device. Standard containers that conform to the concept of random access traversal can be adapted into devices if necessary. Used, located in the namespace boost::iostreams::example.
file device
The file device is located in the header file <boost/iostreams/device/file.hpp>. The file stream based on the standard library provides access to files. The usage is similar to the fstream of the standard library. The iostreams library provides three file devices: file_source, file_sink and file, which are source devices, receiving devices and positionable devices respectively.
These three file devices are actually char type specializations of basic_file_source, basic_file_sink and basic_file. The core class is basic_file.
template<typename Ch>
class basic_file
{
public:
typedef Ch char_type; //字符类型
struct category //设备的分类
: public seekable_device_tag,
public closable_tag,
public localizable_tag
{ };
basic_file(const std::string& path,openmode mode);
std::streamsize read(char_type* s,std::streamsize n);
std::streamsize write(const char_type* s,std::streamsize n);
bool putback(char_type c);
std::streampos seek(stream_offset off,seekdir way,
openmode which = in | out);
void open(const std::string& path,openmode mode);
bool is_open() const;
void close();
private:
struct impl
{
impl(const std::string& path,openmode mode) //内部实现类
{ file_.open(path.c_str(),mode); }
~impl(){
if (file_.is_open()) file_.close();}
std::basic_filebuf file_;
};
shared_ptr<impl> pimpl;
}basic_file uses shared_ptr and pimp to wrap std::basic_filebuf, so users do not need to worry about the opening and closing of files, shared_ptr automatically manages the life cycle.
string str("file device");
file_sink fsink("test.txt");
io::copy(
make_iterator_range(str),
fsink);
file_source fsrc("test.txt");
io::copy(fsrc,cout);empty device
Null devices are located in the header file <boost/iostreams/device/null.hpp>. They are specific applications of the null object mode and are devices that do not perform any actions. The iostreams library provides two null devices: null_source and null_sink, which are sources respectively. equipment and receiving equipment.
null_source and null_sink are actually char type specializations of basic_null_source and basic_null_sink. They only have different input and output modes. In the end, they implement the same class basic_null_device:
template<typename Ch,typename Mode>
class basic_null_device{
public:
typedef Ch char_type;
struct category
: public Mode,
public device_tag,
public closable_tag
{ };
std::streamsize read(Ch*,std::streamsize){
return 0;}
std::streamsize write(const Ch*,std::streamsize n){
return n;}
std::streamsize seek(...){
return -1;}
void close(){}
void close(BOOST_IOS::openmode){}
}All member functions of the null device do not perform any operations or process data. Users cannot obtain any data from null_source because the read() function always returns 0 bytes of data. Users can write any data to null_sink, but the data will Completely ignored. Empty devices are generally used in certain specific situations, such as not caring about the whereabouts of data or prohibiting reading and processing of data.
filter
A filter is a special device that does not have the read and write functions of the source device or the receiving device. It can only allow data to flow through, but it can perform any operation on the data that flows through and complete certain special functions.
To determine whether a class is a filter, you can use the meta-function is_filter<T>.
Filter is the conceptual class of the filter, located in the header file <boost/iostreams/concepts.hpp>.
Like device, filter also has two template parameters. The parameter Mode determines the mode of the device. The parameter Ch is the character type that the device can handle. It can also be obtained using the corresponding meta-functions char_type_of and category_of.
Filter also defines several specializations, which represent sub-concepts of some devices, such as:
typedef filter<input> input_filter; //输入过滤器
typedef filter<output> output_filter; //输出过滤器
typedef filter<seekable> seekable_filter;//可定位过滤器
typedef filter<dual_use> dual_use_filter;//两用过滤器iostreams provides a large number of predefined filters and user-customized filters. The following will first introduce the concepts of device chains and pipelines, and then introduce several commonly used filters.
Equipment chains and pipelines
The concept of device chain was briefly mentioned earlier. The iostreams library draws on the concept and form of "pipeline" in UNIX. Data flows from one device to another through the pipe. Overload operator | and use "|" to connect multiple filters. Chained, usually ending with a device, forming a device chain.
If the end point of the chain is not a device, then the chain is called an "incomplete chain" and cannot be used for IO operations. Otherwise, it is called a "complete chain".
chain is used to manage the device chain. It is located in the header file <boost/iostreams/chain.hpp>. std::list is used internally to save copies of all devices. The member function component_type() returns the type information of the nth device, and component( ) returns the device as a pointer. Note that the template parameters of component() cannot be automatically derived and must be specified manually.
The chain can be constructed as an empty chain at the beginning, or a device or multiple devices connected by pipes can be passed in. The member function push() can be used to append a device or stream to the end of the chain at any time, and pop() can delete a device or stream from the end of the chain. equipment.
The pipeline function of the iostreams library is located in the header file <boost/iostreams/pipeline.hpp>, which provides an auxiliary class pipeline and an overloaded "operator |" operator, allowing users to simply push devices into the chain using "syntactic sugar" .
However, references wrapped by the boost.ref library cannot use the pipeline function, because what is returned after being wrapped by the ref library is the reference_wrapper type, and there is no "operator |" overload.
count filter
The counter filter is located in the header file <boost/iostreams/filter/counter.hpp> and is a dual_use filter, which means it can be used as both input and output. It is a "transparent" filter , does not make any changes to the data flowing through, and only counts the number of characters and lines, similar to the UNIX tool wc.
The counter is a simple filter. As long as you add it to the device chain and use the io::copy algorithm to let the data flow through it, you can use the member functions characters() and lines() to obtain statistical data.
string str("counter\nfilter\n");
filtering_ostream out(counter() | null_sink());
io::copy(boost::make_iterator_range(str),out);
auto pc = out.component<counter>(0); //获得计数过滤器指针
assert(pc->characters() == 15);
assert(pc->lines() == 2);newline filter
The newline filter newline_filter is used to solve the problem that text files in different operating systems use different line terminators: UNIX uses '\n', Apple II and older Macs (System1 until OS9) use '\r' , while DOS/Windows uses '\r\n'.
newline_filter is located in the header file <boost/iostream/filter/newline.cpp> and is a dual_use filter. Its constructor requires specifying the target format constant for conversion. Several newline character constants are defined in the namespace boost::iostreams::newline:
namespace newline {
const char CR = 0x0D;
const char LF = 0x0A;
const int posix = 1;
const int mac = 2;
const int dos = 4;
const int mixed = 8;
}Instructions:
string str("abcdef\n" "12345\n" "aochijk\n");//UNIX换行格式
string result;
filtering_ostream out(
newline_filter(newline::mac) | //转换到旧式mac格式
io::back_inserter(result));
io::copy(boost::make_iterator_range(str),out);<boost/iostream/filter/newline.cpp> also provides a filter for checking newline characters - newline checker (newline_checker) . It does not modify the text. After filtering the data, you can use the member function to obtain the text. Line break related information, usage is basically the same as newline_filter:
filtering_ostream out(
newline_checker() | io::null_sink());
io::copy(boost::make_iterator_range(str),out);
assert(out.component<newline_checker>(0)->is_posix());Regular Expression Filter (I)
The iostreams library provides two regular expression filters. The first is regex_filter, which searches for matching regular expressions in the stream. If the match is successful, the text is replaced. It is located in the header file <boost/iostreams/filter/regex.hpp>. Similar to the UNIX tool sed.
To use regex_filter, the boost.regex library must be linked.
basic_regex_filter is the core class of the regular expression filter. It uses the regex library to perform regular expression matching, so a regex object must be passed in during construction. The second parameter of the constructor is the target text to be replaced, and C characters can be used directly. String or simple text in the form of a standard string.
basic_regex_filter also has a constructor that uses the boost.function library and accepts any single-parameter function or function object used for formatting. This function accepts match_results of regular expression matching results and returns the processed string.
The usage of regex_filter is basically the same as the regex_replace() function of the regex library, but the method is in the form of stream processing.
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/array.hpp> //预定义的数组设备头文件
#include <boost/iostreams/filtering_stream.hpp> //过滤流头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/filter/regex.hpp>
#include <string>
#include <boost/regex.hpp>
#include <iostream>
namespace io = boost::iostreams;
using namespace io;
using namespace std;
int main()
{
boost::regex reg("a.c");
string str("abcdef aochijk");
string result;
filtering_ostream out( //输出过滤流，需连接sink设备作为链结束
regex_filter(reg,"test") | //正则表达式过滤器
io::back_inserter(result)); //适配标准容器作为接收设备
io::copy(
boost::make_iterator_range(str),
out);
std::cout << result << endl;
return 0;
}
//g++-4.8 boost_iostream.cpp -o boost_io -lboost_regex-mt -lboost_systemIn order to use the formatting function, the regular expression must be modified, add the definition of subexpression, and write a function with the parameter regex_filter::match_type to process the matching result:
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/array.hpp> //预定义的数组设备头文件
#include <boost/iostreams/filtering_stream.hpp> //过滤流头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/filter/regex.hpp>
#include <string>
#include <boost/regex.hpp>
#include <iostream>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
int main()
{
boost::regex reg("a(.)c");
std::string str("abcdef aochijk");
std::string result;
//lambda表达式定义格式化函数，使用第1个子表达式
auto formatter = [](const regex_filter::match_type& match)
{
return std::string("test-") + match[1] + "-";
};
filtering_ostream out( //输出过滤流，需连接sink设备作为链结束
regex_filter(reg,formatter) | //正则表达式过滤器
io::back_inserter(result)); //适配标准容器作为接收设备
io::copy(
boost::make_iterator_range(str),
out);
std::cout << result << std::endl;
return 0;
}
//g++ boost_iostream.cpp -std=c++0x -o boost_io -lboost_regex-mt -lboost_system
//g++4.8的C++11进行编译会编译失败，原因未知Regular Expression Filter (II)
grep_filter is another regular expression filter in the iostreams library. It provides a function similar to the UNIX tool grep. It can capture matching lines in the stream and is located in the header file <boost/iostreasms/filter/grep.hpp>.
basic_grep_filter is the core class of grep_filter. The main function of grep_filter is focused on its constructor. It uses regular expressions to match strings in the stream and determines the matching method based on options. If it is whole_line, then matching lines are retained. If it is invert, then unmatched lines are retained. rows, the last number of rows extracted can be obtained using the member function count().
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/array.hpp> //预定义的数组设备头文件
#include <boost/iostreams/filtering_stream.hpp> //过滤流头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/filter/grep.hpp>
#include <string>
#include <boost/regex.hpp>
#include <iostream>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
int main()
{
boost::regex reg("a(.)c");
std::string str("abcdef\n" "12345\n" "aochijk\n");
std::string result;
filtering_ostream out( //输出过滤流，需连接sink设备作为链结束
grep_filter(reg) | //正则表达式过滤器，缺省为whole
io::back_inserter(result)); //适配标准容器作为接收设备
io::copy(
boost::make_iterator_range(str),
out);
std::cout << result << std::endl;
return 0;
}#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/array.hpp> //预定义的数组设备头文件
#include <boost/iostreams/filtering_stream.hpp> //过滤流头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/filter/grep.hpp>
#include <string>
#include <boost/regex.hpp>
#include <iostream>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
int main()
{
boost::regex reg("a(.)c");
std::string str("abcdef\n" "12345\n" "aochijk\n");
std::string result;
filtering_ostream out( //输出过滤流，需连接sink设备作为链结束
grep_filter(reg,regex_constants::match_default,grep::invert) | //正则表达式过滤器，缺省为whole
io::back_inserter(result)); //适配标准容器作为接收设备
io::copy(
boost::make_iterator_range(str),
out);
std::cout << result << std::endl;
return 0;
}compression filter
The iostreams library supports three compression algorithms, namely zlib, gzip and bzip2, which must be used with the corresponding zlib and bzip2 libraries.
For example, the zlib compression algorithm is located in the header file <boost/iostreams/filter/zlib.hpp>
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/array.hpp> //预定义的数组设备头文件
#include <boost/iostreams/filtering_stream.hpp> //过滤流头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/filter/zlib.hpp>
#include <iostream>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
int main()
{
std::string str("12345678j 12345678");
std::string zip_str,unzip_str;
filtering_ostream zout( //输出过滤流，需连接sink设备作为链结束
zlib_compressor() | //使用压缩过滤器输出
io::back_inserter(zip_str));//适配标准容器作为接收设备
io::copy(
boost::make_iterator_range(str),
zout);
std::cout << zip_str << std::endl;
filtering_ostream uzout(
zlib_decompressor() |
io::back_inserter(unzip_str));
io::copy(
boost::make_iterator_range(zip_str),
uzout);
std::cout << unzip_str << std::endl;
assert(unzip_str == str);
return 0;
}
//g++-4.8 boost_iostream.cpp -o boost_io -lboost_iostreams-mt -lzflow
Devices and filters are just isolated objects, and streams are the link that connects them. Through streams, data can be transferred from one device to the next, and data processing can be realized during the transfer process.
elementary stream
There are two basic streams in the iostreams library: stream and stream_buffer. The functions and behaviors of the two are very similar. Let’s discuss stream as an example. It is located in the header file <boost/iostreams/stream.hpp>. Stream is a subclass of the standard library stream and
has All functions of the standard IO stream, such as get(), read() and other operations, but unlike the streams of the standard library, the input stream and the output stream are not distinguished by name. Its direction depends on its associated device T - use The source device is the input stream, and the receiving device is the output stream.
Stream can open the device directly during construction, or call open() to open the device later. Their function parameters accept the example of the device in the single-parameter form, and accept the N parameters required to construct the device in the multi-parameter form. An instance of the device is created internally by stream. If the stream has been associated with an actual device, the member function is_open() will return true. At this time, you can use the overloaded operator* and operator-> to obtain the device reference within the stream.
filter stream
The filter flow is an enhanced version of the basic flow. It can not only connect devices, but also filters and chains, so it is more versatile and more powerful.
There are two template classes for filtering streams in the iostreams library: filtering_stream and filtering_streambuf. Their functions and behaviors are similar. Let’s discuss filtering_stream as an example. The interface of filtering_stream located in the header file <boost/iostreams/filtering_stream.hpp> is closer to chain. The main template parameter is Mode,
specified The mode for filtering streams. The constructor of the filter stream can pass in multiple device objects and use pipe characters ("|") to connect them to form a device chain. If the end point is a receiving device, then it is equivalent to an enhanced output stream and can be written Input data, perform filtering operations when writing data, and vice versa is an enhanced input stream. It should be noted that the pipeline operation only targets the device and cannot connect the stream. To add the stream object to the filter stream, you need to use the member function push().
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/array.hpp> //预定义的数组设备头文件
#include <boost/iostreams/filtering_stream.hpp> //过滤流头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/filter/counter.hpp>
#include <iostream>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
int main()
{
char ar[10] = "abcd";
stream<array_source> sa(ar); //定义一个字符数组输入流
counter ct;
filtering_istream in; //定义一个空的输入过滤流
assert(in.empty());
in.push(ref(ct)); //加入计数过滤器，使用ref包装，链不完整
assert(!in.is_complete());
in.push(sa); //加入输入流，链完整
assert(in.is_complete());
filtering_ostream out(std::cout);
io::copy(in,out);
assert(ct.characters() == 10);
return 0;
}Stream processing function
Devices, filters and streams are the three major elements of the iostreams library. Although streams connect devices and filters, an external driver is needed to actually flow data - a stream processing function.
The iostreams library provides several stream processing functions, the most important of which is io::copy(), which drives the entire data stream - reading characters from the source device or stream, and then writing them to the receiving device or stream. Finally closes both devices and returns the number of characters processed.
io::copy() is located in the header file <boost/iostreams/copy.hpp>. It has four overloaded forms to deal with the situation where src and sink are devices or streams. The previous code example uses io:: multiple times. copy() operates on stream objects. In fact, it can also directly operate on devices:
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/array.hpp> //预定义的数组设备头文件
#include <boost/iostreams/filtering_stream.hpp> //过滤流头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/filter/counter.hpp>
#include <iostream>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
int main()
{
char ar[] = "abcd123";
stream<array_source> in(ar); //源设备
io::copy(in,std::cout); //从源设备到输出流
std::string result;
in.open(ar); //再次打开源设备
io::copy(in,io::back_inserter(result));//从源设备到接收设备
return 0;
}In addition to io::copy(), there are other stream processing functions in the iostreams library, most of which are located in the header file <boost/iostreams/operations.hpp>. Commonly used functions include the following:
- get(d): Get a character from the source device or stream
- read(d,s,n): Get multiple characters from the source device or stream
- put(d,c): Write a character to the receiving device or stream
- write(d,s,n): Write multiple characters to the receiving device or stream
- seek(d,off,way): Randomly access a device or stream
- flush(d): Flush the device or the entire stream, clear the buffer
- close(d): close the device or stream
These functions mainly come into play when writing custom devices or filters
Customized equipment
Custom source device
To write a device, you first need to meet the concept of the device. The header file <boost/iostreams/concepts.hpp> defines several specializations to represent some sub-concepts of the device, including sources defined to facilitate user expansion. The statement is as follows:
typedef device<input> source; //char源设备As long as the custom device public inherits source, the device will automatically meet the concept of source device. Of course, it is not necessary to use source as the base class (such as the iostreams library's own array device and file device).
The source device is used for input, so it needs to implement a read() member function of the following form:
//最多读取n个字符到缓冲区s中，然后返回读取的字符数表示读取成功，返回EOF(-1)则表示已经读取完毕
std::streamsize read(char_type* s,std::streamsize n);Example 1: Random number source device
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/concepts.hpp>
#include <boost/random.hpp>
#include <algorithm>
#include <iostream>
#include <string>
#include <time.h>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
class rand_source:public io::source //定义随机数源设备
{
private:
std::streamsize count; //需要产生的随机数数量
public:
rand_source(int c):
count(c){}
//核心函数，实现源设备的流读取功能
std::streamsize read(char_type* s,std::streamsize n)
{
//最多读取n个字符到缓冲区s中，返回读取的字符数
auto read_count = (std::min)(n,count);
if(read_count)
return EOF;
for(std::streamsize i = 0;i<read_count;++i)
*s++ = rand()();
count -= read_count; //当前剩余的读取数量
return read_count; //返回读取的字符数
}
private:
//使用变量发生器组合rand48和uniform_smallint
typedef boost::variate_generator<boost::rand48,boost::uniform_smallint<> > rand_t;
static rand_t& rand()
{
static rand_t r(boost::rand48(time(0)),boost::uniform_smallint<>('0','z'));
return r;
}
};
int main()
{
std::string out;
io::copy(rand_source(20), //直接使用源设备
io::back_inserter(out)); //输出到字符串
return 0;
}Example 2: Improved random number source device
This example changes the character type of stream processing from char to unsigned char. Because the source is a char device, it cannot be inherited from it. It must be directly specialized from the device, and other logic remains unchanged:
class rand_source:public io::device<input,unsigned char>
{..}; //实现代码同前Since the devices on the stream processing device chain must use consistent character types, a matching standard container block needs to be defined, which is a specialization of basic_string:
typedef std::basic_string<unsigned char> block;use:
block out; //注意不能使用std::string，字符类型不同
io::copy(rand_source(10), //直接使用源设备
io::back_inserter(out)); //输出到字符串Customized receiving equipment
Custom receiving devices must also meet the device definition, but for the receiving device, its mode must be output, and its convenient specialized form is sink:
typedef device<output> sink; //char接收设备For the receiving device, you need to write the write() member function in the following form:
//从缓冲区s中最多获取n个字符，然后写入某个地方，最后返回写入的字符数
std::streamsize write(const char_type* s,std::streamsize n);A simple receiving device cout_sink implementation:
class cout_sink:public io::sink
{
public:
std::streamsize write(const char_type* s,std::streamsize n)
{
std::cout << std::string(s,n) << std::endl;
return n;
}
};Custom filters
Filter implementation principle
Writing filters must conform to the concept of filter, which is located in the header file <boost/iostreams/concepts.hpp>. Before writing filters, first understand several auxiliary classes for writing filters
provided in the iostreams library , which can make it easier for users to write Filters include the following:
- aggregate_filter: dual-purpose filter, receiving all data at once
- basic_line_filter: dual-purpose filter, extracting one line of data at a time
- basic_stdio_filter: dual-purpose filter, used to adapt to standard input and output streams
- symmetric_filter: dual-purpose filter with internal buffer.
The first three are simple to use. Just implement the pure virtual function do_filter() and write appropriate processing logic in it. The last application is more complicated and will not be introduced.
Custom filters support the pipe operator of iostreams, which allows the filter to work better with other devices and streams. The iostreams library specifically provides a macro BOOST_IOSTREAMS_PIPABLE, which can add pipeline functionality to the filter with just one line of code:
#define BOOST_IOSTREAMS_PIPABLE(filter,arity)BOOST_IOSTREAMS_PIPABLE is usually used after the filter is defined. The first parameter is the name of the filter, and the second parameter is the number of filter template parameters. If the filter is not a template class, the value should be 0.
aggregate_filter
aggregate_filter is located in the header file <boost/iostreams/filter/aggregate.hpp>. It uses the template method pattern to complete most of the reading and writing work required by the filter. It only needs to implement the core function of filtering do_filter(). It has two parameters. The first src is all the characters received by the filter. These characters can be processed in any way, and then the processing results are added to the second parameter dest.
Another virtual function do_close() is called when the filter is closed and can be implemented with the default empty value.
The previous regex_filter is actually implemented using aggregate_filter.
Demonstration, use aggregate_filter to write a hexadecimal-encoded output filter hex_filter, which utilizes the hex algorithm of the boost.algorithm library:
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/concepts.hpp>
#include <boost/iostreams/filter/counter.hpp>
#include <boost/iostreams/filter/aggregate.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/algorithm/hex.hpp>
#include <iostream>
#include <string>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
class hex_filter final:public io::aggregate_filter<char> //定义随机数源设备
{
private:
typedef io::aggregate_filter<char> super_type; //简化类型定义
typedef super_type::vector_type vector_type;
virtual void do_filter(
const vector_type& src,vector_type& dest)
{
boost::algorithm::hex(src,std::back_inserter(dest));
}
};
BOOST_IOSTREAMS_PIPABLE(hex_filter,0);
int main()
{
char str[] = "313";
std::string result;
filtering_ostream out( //输出流
hex_filter() | //hex过滤器
counter() | //统计流过的字符数
io::back_inserter(result));
io::copy(array_source(str,str + 3),out);
assert(result.size() == 6); //hex编码后长度变为两倍
assert(out.component<counter>(1)->characters() == 6);
return 0;
}basic_line_filter
basic_line_filter is located in the header file <boost/iostreams/filter/line.hpp>. Its usage is similar to aggregate_filter, but it only filters one line of characters. This line of characters is processed by the pure virtual function do_filter(), and then the processed new line is returned.
In order to facilitate the use of basic_line_filter, two specializations of line_filter and wline_filter are provided:
typedef basic_line_filter<char> line_filter;
typedef_basic_line_filter<wchar_t> wline_filter;#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/concepts.hpp>
#include <boost/iostreams/filter/counter.hpp>
#include <boost/iostreams/filter/line.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <iostream>
#include <string>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
class bracket_line_filter final:public io::basic_line_filter<char> //定义随机数源设备
{
private:
typedef io::basic_line_filter<char> super_type; //简化类型定义
typedef typename super_type::string_type string_type;
virtual string_type do_filter(const string_type& line)
{
return "<" + line + ">"; //简单处理返回
}
};
BOOST_IOSTREAMS_PIPABLE(bracket_line_filter,0);
int main()
{
std::string str("The\n" "Phantom\n" "Pain");
filtering_istream in( //输入流
bracket_line_filter() |
boost::make_iterator_range(str));
io::copy(in,std::cout);
return 0;
}Handmade filters
To write a filter manually, you only need to write out the necessary elements of the filter such as char_type, category, read(), write(), close() implemented by the original auxiliary class yourself.
For multi-character filters, we need to implement read and write functions in the following form:
template<typename Source>
std::streamsize read(Source& src,char_type* s,std::streamsize n)
{
//从src读取字符处理，处理后最多写入n个字符到缓冲区s，返回读取的字符数或者EOF
}
template<typename Sink>
std::streamsize write(Sink& dest,const char_type* s,std::streamsize n)
{
//从缓冲区s最多读取n个字符处理，将处理后的字符写到des，返回已处理的字符数
}Demonstration, use the SHA1 utility class of the boost.uuid library to implement an output filter that calculates the SHA1 digest:
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/concepts.hpp>
#include <boost/iostreams/filter/counter.hpp>
#include <boost/iostreams/filter/aggregate.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/uuid/detail/sha1.hpp>
#include <iostream>
#include <string>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
class sha1_filter final //不是模板类，也不使用任何的概念类
{
public:
typedef char char_type; //字符类型定义
struct category: //过滤器分类定义
output, //输出模式
filter_tag, //过滤器设备
multichar_tag, //处理多字符
closable_tag //可关闭
{ };
private:
boost::uuids::detail::sha1 sha;
public:
template<typename Sink> //输出模式需要实现write()函数
std::streamsize write(Sink&,const char_type* s,std::streamsize n)
{
sha.process_bytes(s,n); //处理所有输入字符序列
return n; //返回处理的字符数，但暂不输出摘要结果
}
template<typename Sink>
void close(Sink& snk) //过滤器关闭时给出摘要结果
{
unsigned int digest[5];
sha.get_digest(digest);
char_type* p = reinterpret_cast<char_type*>(digest);
io::write(snk,p,sizeof(digest)/sizeof(char_type)); //写入接收设备
}
};
BOOST_IOSTREAMS_PIPABLE(sha1_filter,0);
int main()
{
char str[] = "The Evil Within";
std::string result;
filtering_ostream out( //输出流
sha1_filter() | //sha1过滤器
io::back_inserter(result));
io::write(out,str,3); //使用write()函数向流写入数据
assert(result.empty()); //没有输出结果
io::write(out,str + 3,3); //继续使用write()函数向流写入数据
assert(result.empty()); //没有输出结果
io::close(out);
assert(result.size() == 20); //得到摘要结果
return 0;
}The io::copy() function is not used here, because io::copy() not only copies the stream data but also automatically closes the stream, so the stream cannot be used after copying, and the write() function does not close the stream, so it can be used more Write data once, and finally close the stream manually to get the results.
sha1_filter can also simplify the code using another conceptual class multichar_output_filter, which only defines char_type and category.
To implement the copy algorithm without shutting down the device
, the io::copy() algorithm cannot be used for multi-segment input filters such as sha1_filter. Write() is obviously not as convenient to use as io::copy(), so you can imitate it and write one that does not The copy version of the copy_no_close() function that automatically closes the device.
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/copy.hpp> //io::copy算法头文件
#include <boost/iostreams/filtering_stream.hpp>
#include <iostream>
#include <string>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
template<typename Source,typename Sink>
std::streamsize copy_no_close_impl(Source src,Sink snk,std::streamsize buffer_size)
{
return io::detail::copy_operation<Source,Sink>(
src,snk,buffer_size)(); //调用copy操作，不关闭设备
}
template<typename Source,typename Sink>
std::streamsize copy_no_close(const Source& src,Sink& snk,
std::streamsize buffer_size = io::default_device_buffer_size
BOOST_IOSTREAMS_ENABLE_IF_STREAM(Source)
BOOST_IOSTREAMS_ENABLE_IF_STREAM(Sink))
{
return copy_no_close_impl(io::detail::wrap(src),io::detail::wrap(snk),buffer_size);
}
int main()
{
char str[] = "The Evil Within";
stream<array_source> in(str,str+15);
std::string result;
filtering_ostream out( //输出流
io::back_inserter(result));
copy_no_close(in,out);
assert(in.is_open());
io::close(in);
io::close(out);
return 0;
}Combination equipment
The iostreams library provides some template classes and functions that can combine simple and easy-to-implement devices or filters to form usable new devices.
combine
The combination template class can "parallel" combine a pair of source/receiver devices or input/output filters to form a new device or filter. The new device is a bidirectional device that operates on two independent character sequences, using the former for input and the latter for output. The iostreams library provides the factory function combine() to simplify class construction, located in the header file <boost/iostreams/combine.hpp>
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/combine.hpp>
#include <iostream>
#include <string>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
int main()
{
char src[] = "12345678"; //输入字符序列
std::string result; //输出字符序列
typedef io::combination<array_source, //组合设备类型定义
back_insert_device<std::string> > bi_dev;
assert(is_device<bi_dev>::value); //检查是否是设备
assert((is_same< //检查是否是双向模式
mode_of<bi_dev>::type,bidirectional>::value));
bi_dev dev = io::combine( //使用函数生成组合设备
array_source(src),
io::back_inserter(result));
io::write(dev,src,2); //向输出序列写入数据
assert(result.size() == 2);
return 0;
}compose
Compose is another tool for combining devices or filters. It "concatenates" two devices, and the data flows through the new device in order (output mode) or reverse order (input mode). It is somewhat similar to the pipe character and is located in the header file. <boost/iostreams/compose.hpp>
The template class provided by compose is composite. Its first template parameter is a filter, and the second template parameter is a filter, device or stream. Similar to std::pair, you can use first() and second() to obtain the object it combines, and the factory function compose() is used to automatically deduce the parameters to generate a new device object.
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/compose.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/regex.hpp>
#include <boost/regex.hpp>
#include <iostream>
#include <string>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
int main()
{
std::string str("abcdef aochijk"); //输入字符串
io::copy(
boost::make_iterator_range(str),
compose(regex_filter(regex("a.c"),"test"),//正则表达式过滤器
std::cout) //标准输出流
);
return 0;
}
// g++-4.8 -std=c++11 boost_iostream.cpp -o boost_io -lboost_regex -lboost_system
// 重新安装了boost库后使用g++-4.8的C++11编译成功invert
invert uses the function of compose to adapt a filter and invert its mode - that is, turn the input filter into an output filter or vice versa, located in the header file <boost/iostreams/invert.hpp> invert uses the template class
inverse To achieve mode reversal, Yuan calculates the mode of the adapted filter, then changes it to the opposite mode, defines a source or sink device internally, uses compose to combine the filter and device, and then completes reading and writing. The factory function invert() is used to automatically deduce parameters to generate an inverted new device object.
Although invert only requires one template parameter and is simple to use, its actual use will be limited. For example, it cannot be used for dual-use filters, because dual-use filters themselves support input and output, and using invert will cause compilation errors ( The mode cannot be inverted). In addition, filters that have read and write operations when closed, such as the sha1_filter implemented earlier, cannot use invert, because the device composed inside the template class inverse is only a temporary object, which is only valid during read and write operations.
A simple output filter null_filter:
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/invert.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <iostream>
#include <string>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
class null_filter final
{
public:
typedef char char_type;
struct category:
output,
filter_tag,
multichar_tag,
closable_tag
{ };
public:
template<typename Sink>
std::streamsize write(Sink& snk,const char_type* s,std::streamsize n)
{
io::write(snk,s,n);
return n;
}
template<typename Sink>
void close(Sink& snk){}
};
BOOST_IOSTREAMS_PIPABLE(null_filter,0); //支持管道符操作
BOOST_IOSTREAMS_PIPABLE(inverse,1); //为inverse添加管道符操作
int main()
{
char str[] = "Ground Zero";
std::string result;
filtering_istream in(
invert(null_filter()) |
array_source(str));
io::copy(in,io::back_inserter(result));
return 0;
}restrict
restrict is used to add a constraint to the adapted device or filter, which limits access to only one subrange. It is located in the header file <boost/iostreams/restrict.hpp> because restrict is a
keyword in the C99 standard (in C++ (not in), in order to maintain compatibility, it provides another name slice, located in the header file <boost/iostreams/slice.hpp>
template class restriction to implement access restrictions, the key lies in its constructor, the parameter off specifies the offset position, len Specify the offset length and limit access to the interval of [start + off, start + off + len], which is somewhat similar to C's file operation function fseek. The factory function restrict() or slice() is used to automatically deduce the new device object after parameter generation constraints.
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/restrict.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/counter.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <iostream>
#include <string>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
BOOST_IOSTREAMS_PIPABLE(restriction,1);
int main()
{
char str[] = "1234";
std::string result;
filtering_istream in(
restrict(counter(),1,2) |
array_source(str));
io::copy(in,io::back_inserter(result));
assert(result == "23");
return 0;
}Restrict is most commonly used for file stream access. It specifies a specific location for reading and writing files, which is more convenient than calling seekg()/seekp() of the standard library.
tee
The tee tool provides the function of separating stream output. It can establish a branch of the stream and send the upstream data to another receiving device at the same time. With compose, it can perform two or more different stream processing on a sequence at the same time. It is located in The header file <boost/iostreams/tee.hpp>
example uses compose to create two receiving devices and uses tee to split the data, so that the data is output to the standard stream and standard container at the same time:
#include <boost/iostreams/stream.hpp> //基本流处理所需的头文件
#include <boost/iostreams/device/back_inserter.hpp> //接收设备适配头文件
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/tee.hpp>
#include <boost/iostreams/compose.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/regex.hpp>
#include <boost/iostreams/filter/zlib.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <iostream>
#include <string>
namespace io = boost::iostreams;
using namespace io;
using namespace boost;
int main()
{
std::string str("abcdef aochijk");
std::string result;
filtering_ostream out( //输出流
tee(
compose(regex_filter(regex("a.c"),"test"),std::cout),
compose(zlib_compressor(),io::back_inserter(result))
)
);
io::copy(boost::make_iterator_range(str),out);
return 0;
}
//g++-4.8 -std=c++11 boost_iostream.cpp -o boost_io -lboost_regex -lboost_iostreams -lboost_system -lz
//和之前只使用regex的那次编译不一样，必须加上-lboost_iostreams，否则编译失败Comparison with iterators
Stream processing is very similar to how iterators handle data sequences:
- Both use the copy algorithm to traverse a source sequence and then write the processed data to another sequence.
- The mode of stream is very close to the classification of iterator, and can also be divided into readable input mode and writable output mode.
- Iterators can be nested and combined to provide complex functions, while streams can filter and process data in the form of device chains.
- After iterators are adapted to the concept of intervals, filters can also be used to process data like streams.
The difference between stream processing and iterators is also obvious:
- Iterators are based on the iterator design pattern, do not use virtual functions, and complete data processing by rewriting the core operations of the iterator; iostreams is based on the established standard stream framework, using stream processing ideas, and using concepts such as devices and filters. to process data.
- Iterators are generic and can handle any type of data; although iostreams are also generic, they focus more on processing character types.
- Iterators are usually used to process data in memory, while iostreams can also process a wider range of data such as files and sockets.
- Although iterators can also be used in combination, they can only be determined at compile time. Obviously, the device chain without streams can be added and disassembled in a flexible way.