Previous post in our preliminary experience using google benchmark, and in this article we will further understanding of common methods google benchmark.
Citation
Passing parameters to test cases
Before we test cases only accepts a benchmark::State¶meter of type, if we need to pass additional parameters to test it?
For example, if we need to implement a queue, there are ring buffer linked list and two implementation options, and now we want to test the performance of two programs in different situations:
// 必要的数据结构
#include "ring.h"
#include "linked_ring.h"
// ring buffer的测试
static void bench_array_ring_insert_int_10(benchmark::State& state)
{
auto ring = ArrayRing<int>(10);
for (auto _: state) {
for (int i = 1; i <= 10; ++i) {
ring.insert(i);
}
state.PauseTiming(); // 暂停计时
ring.clear();
state.ResumeTiming(); // 恢复计时
}
}
BENCHMARK(bench_array_ring_insert_int_10);
// linked list的测试
static void bench_linked_queue_insert_int_10(benchmark::State &state)
{
auto ring = LinkedRing<int>{};
for (auto _:state) {
for (int i = 0; i < 10; ++i) {
ring.insert(i);
}
state.PauseTiming();
ring.clear();
state.ResumeTiming();
}
}
BENCHMARK(bench_linked_queue_insert_int_10);
// 还有针对删除的测试，以及针对string的测试，都是高度重复的代码，这里不再罗列Obviously, in addition to the above test to test the type and amount of data to be inserted without any distinction, if it can be controlled by passing a large number of parameters can be written less repetitive code.
Write duplicate code is a waste of time, and often means you're doing something stupid, google engineers would have certainly noticed it. Although the test can only accept a benchmark::State¶meter of type, but we can pass arguments to the state object, and then get in a test case:
static void bench_array_ring_insert_int(benchmark::State& state)
{
auto length = state.range(0);
auto ring = ArrayRing<int>(length);
for (auto _: state) {
for (int i = 1; i <= length; ++i) {
ring.insert(i);
}
state.PauseTiming();
ring.clear();
state.ResumeTiming();
}
}
BENCHMARK(bench_array_ring_insert_int)->Arg(10);The above example shows how to transfer and acquisition parameters:
- Passing parameters using
BENCHMARKthe target macroblock generatedArgMethod - Parameters passed in the object will be placed in the internal state stored by the
rangeacquisition method, parameter 0 is an incoming call to the required parameters, the first parameter corresponding to
ArgThe method can only pass a parameter that if one wants to pass multiple parameters? It is also very simple:
static void bench_array_ring_insert_int(benchmark::State& state)
{
auto ring = ArrayRing<int>(state.range(0));
for (auto _: state) {
for (int i = 1; i <= state.range(1); ++i) {
ring.insert(i);
}
state.PauseTiming();
ring.clear();
state.ResumeTiming();
}
}
BENCHMARK(bench_array_ring_insert_int)->Args({10, 10});The example above no practical significance, just to show how to pass multiple parameters, Argsmethod accepts a vector object, so we can use c ++ 11 provides braces simplifies initialization code parameters still get through state.rangemethod, 1 corresponds passed in The second parameter.
It is worth noting, we can only accept integer parameters are passed, if you want to use other types of additional parameter, you need to find some way of another.
Generating a plurality of test cases similar simplification
The ultimate goal of passing parameters to test cases in order to generate more test cases without writing duplicate code, after knowing how to pass parameters you might write:
static void bench_array_ring_insert_int(benchmark::State& state)
{
auto length = state.range(0);
auto ring = ArrayRing<int>(length);
for (auto _: state) {
for (int i = 1; i <= length; ++i) {
ring.insert(i);
}
state.PauseTiming();
ring.clear();
state.ResumeTiming();
}
}
// 下面我们生成测试插入10，100，1000次的测试用例
BENCHMARK(bench_array_ring_insert_int)->Arg(10);
BENCHMARK(bench_array_ring_insert_int)->Arg(100);
BENCHMARK(bench_array_ring_insert_int)->Arg(1000);Here we generate three examples, will produce the following results:
Looks good job, is not it?
Yes, the results are correct, but remember we said before it - do not write duplicate code ! Yes, we manually written above generate use cases, it appeared to avoid duplication.
Fortunately, Argand Argsparameters of the test cases we will use to register in order to generate 用例名/参数new test cases, and returns a pointer to BENCHMARKa pointer macro generates object, in other words, if we want to generate more than just test different parameters, then only chain need to call Argand Argsyou can:
BENCHMARK(bench_array_ring_insert_int)->Arg(10)->Arg(100)->Arg(1000);The results and the same as above.
But this is not the optimal solution, we called Arg still repeat method, if we need more use cases would have had to do a duplication of effort.
In this regard google benchmark also has a solution: we can use the Rangemethod to automatically generate a range of parameters.
Take a look at the prototype Range:
BENCHMAEK(func)->Range(int64_t start, int64_t limit);start indicates the start value of the parameter range, limit value indicating the end of the range, Range is made for a closed interval _ _.
But if we rewrite this code is going to get a false test results:
BENCHMARK(bench_array_ring_insert_int)->Range(10, 1000);
Why is this so? That's because in addition to the default Range start and limit, in the middle of the remaining parameters will be the power of one substrate (base), the base 8 by default, so we'll see 64 and 512, which are square and cube 8.
Want to change this behavior is also very simple, as long as the substrate can be reset by using the RangeMultipliermethod:
BENCHMARK(bench_array_ring_insert_int)->RangeMultiplier(10)->Range(10, 1000);The results are now restored as ever.
Ranges can be processed using the case where a plurality of parameters:
BENCHMARK(func)->RangeMultiplier(10)->Ranges({{10, 1000}, {128， 256}});The first range of a specified range of the first parameter passed the test, while the second pass a second specified range of possible values of the parameters (note that this is not the range).
Equivalent to the following code:
BENCHMARK(func)->Args({10, 128})
->Args({100, 128})
->Args({1000, 128})
->Args({10, 256})
->Args({100, 256})
->Args({1000, 256})Actually specified by the contents of a first range parameter generated in the parameter made behind a Cartesian product.
Use parameter generator
If I want to customize more complex parameters did not rule it? Then you need to implement a custom parameter generator a.
A signature parameter generator as follows:
void CustomArguments(benchmark::internal::Benchmark* b);We calculate the parameters at the generator, and then call the benchmark::internal::Benchmarkobject's methods as Arg or Args passed as parameters to the two.
Then we use the Applymethod to be applied to the generator test case:
BENCHMARK(func)->Apply(CustomArguments);In fact, the principle of this process is not complicated, I do a simple explanation:
BENCHMARKIt is a macro generationbenchmark::internal::Benchmarkobject and returns a pointer to it- To
benchmark::internal::Benchmarkpass parameters required Arg objects and the like Args ApplyApplication of the method will function in the parameter itself- We use the generator in the
benchmark::internal::Benchmarkpointer object Args b passing parameters or the like, at this time point b in fact, our test case
So far Builder is already clear how it works, of course, derived from the above conclusion, we can let Apply to do more things.
Apply look below specific use:
// 这次我们生成100，200，...，1000的测试用例，用range是无法生成这些参数的
static void custom_args(benchmark::internal::Benchmark* b)
{
for (int i = 100; i <= 1000; i += 100) {
b->Arg(i);
}
}
BENCHMARK(bench_array_ring_insert_int)->RangeMultiplier(10)->Apply(custom_args);Custom parameters of the test results:
So far the method of passing parameters to test cases on all introduced over.
Next, I will explain how to write test case template, passing parameters can only solve part of the duplication of code for different types of tests to be test cases with similar methods, using the template will greatly reduce our unnecessary work.