OpenResty can also be used in this way from entry to proficiency in 28-test-nginx?

28 | Can test::nginx still be used like this?

Hello, I am Wen Ming.

In the previous two chapters, you have test::nginxmastered most of the usage methods, and I believe you have been able to understand most of the test case sets in the OpenResty project. This is enough for learning about OpenResty and its surrounding libraries.

But if you are interested in becoming a code contributor to OpenResty, or if you are using in your own project test::nginxto write test cases, then you also need to learn some more advanced and complex usage.

Today's content may be the most "cold" part of this column, because this is the content that no one has ever shared. Take lua-nginx-module, the core module in OpenResty, as an example. There are more than 70 contributors worldwide, but not every contributor has written a test case. Therefore, if you finish today's course, your understanding on test::nginxInternet can definitely enter the top 100 in the world.

debugging in test

First, let's look at some of the simplest and most commonly used primitives by developers, which will be used in normal debugging. Next, let's introduce in turn the usage scenarios of these debugging-related primitives.

ONLY

Many times, we add a new test case based on the original test case set. If the test file contains a lot of test cases, it is obviously time-consuming to run it from beginning to end, especially when you need to modify the test cases repeatedly.

So, is there any way to only run a certain test case you specify? ONLYThis markup makes it easy:

=== TEST 1: sanity
=== TEST 2: get
--- ONLY

The pseudocode above shows how to use this primitive. Put --- ONLYin the last line of the test case that needs to be run separately, then when using prove to run the test case file, all other test cases will be ignored, and only this test will be run.

However, this is only suitable for use when you are doing debugging. Therefore, when the prove command finds the ONLY flag, it will also give a prompt, telling you not to forget to remove it when submitting the code.

SKIP

The requirement corresponding to executing only one test case is to ignore a certain test case. SKIPThis flag is generally used to test functions that have not yet been implemented:

=== TEST 1: sanity
=== TEST 2: get
--- SKIP

As you can see from this pseudocode, its usage is similar to ONLY. Because we are test-driven development, we need to write test cases first; and when implementing collective coding, it may be necessary to delay the implementation of a certain function due to the difficulty of implementation or the relationship of priority. Then at this time, you can skip the corresponding test case set first, and then remove the SKIP mark after the implementation is completed.

LAST

Another commonly used flag is LASTthat its usage is also very simple, the test case set before it will be executed, and the latter will be ignored:

=== TEST 1: sanity
=== TEST 2: get
--- LAST
=== TEST 3: set

You may be wondering, I can understand ONLY and SKIP, but what is the use of the LAST function? In fact, sometimes your test cases have dependencies, and you need to execute the first few test cases before the subsequent tests are meaningful. Then, if you go to debug in this case, LAST is very useful.

test plan

Of test::nginxall the primitives, planit's the most maddening and the hardest to understand. It is derived from the perl Test::Planmodule , so the documentation is not test::nginxin and it is not easy to find its explanation, so I put it in the front position to introduce. I have seen several OpenResty code contributors fall into this pit, and they can't even climb out.

Here is an example, you can see a similar configuration at the beginning of each file in the official OpenResty test set:

plan tests => repeat_each() * (3 * blocks());

The meaning of plan here is how many detection items should be done according to the plan in the entire test file. If the results of the final run do not match the plan, the entire test fails.

Take this example, if the value repeat_eachof is 2 and there are 10 test cases, then the value of plan should be 2 x 3 x 10 = 60. I guess the only thing you don't understand here is the meaning of the number 3, it looks like a magic number!

Don't worry, let's continue to look at the example, and you will understand it in a while. Let me talk about it first, can you figure out what is the correct value of plan in the following test case?

=== TEST 1: sanity
--- config
    location /t {
        content_by_lua_block {
            ngx.say("hello")
        }
    }
--- request
GET /t
--- response_body
hello

I believe everyone will conclude that plan = 1, since only is response_bodychecked .

but it is not the truth! The correct answer is, plan = 2. why? Because there is a check hidden test::nginxin , that is --- error_code: 200, it detects whether the HTTP response code is 200 by default.

Therefore, the above magic number 3, the real meaning is that each test is explicitly detected twice, such as body and error log; at the same time, the response code is implicitly detected.

Since this place is too error-prone, my suggestion is that you use the following method to directly close the plan:

use Test::Nginx::Socket 'no_plan';

If it cannot be closed, for example, if the plan is inaccurate in the official test set of OpenResty, it is recommended that you do not go into the cause, just increase or decrease the number in the expression of the plan:

plan tests => repeat_each() * (3 * blocks()) + 2;

This is also the official method used.

preprocessor

We know that there may be some common settings between different test cases in the same test file. If the settings are repeated in each test case, the code will appear redundant, and it will be more troublesome to modify later.

At this time, you can use add_block_preprocessorthe command to add a piece of perl code, such as the following:

add_block_preprocessor(sub {
    my $block = shift;

    if (!defined $block->config) {
        $block->set_value("config", <<'_END_');
    location = /t {
        echo $arg_a;
    }
    _END_
    }
});

This preprocessor will add a section of config configuration for all test cases, and the content inside is location /t. In this way, in your subsequent test cases, you can omit config and access it directly:

=== TEST 1:
--- request
    GET /t?a=3
--- response_body
3

=== TEST 2:
--- request
    GET /t?a=blah
--- response_body
blah

custom function

In addition to adding perl code in the preprocessor, you can run_testsalso add perl functions at will before primitives, which is what we call custom functions.

The following is an example, which adds a function to read files, and combines with evalthe command to realize the function of POST files:

sub read_file {
    my $infile = shift;
    open my $in, $infile
        or die "cannot open $infile for reading: $!";
    my $content = do { local $/; <$in> };
    close $in;
    $content;
}

our $CONTENT = read_file("t/test.jpg");

run_tests;

__DATA__

=== TEST 1: sanity
--- request eval
"POST /\n$::CONTENT"

out of order

In addition to the above points, test::nginxthere is also a little-known pit: by default, test cases are executed out of order and randomly, instead of in accordance with the order and number of test cases.

Its original intention is to test out more problems. After all, after each test case runs, the Nginx process will be shut down and a new Nginx will be started for execution. The results should not be related to the sequence.

For underlying projects, this is true. However, for projects at the application layer, persistent storage such as databases exists externally. Out-of-order execution at this time will lead to wrong results. Since each time is random, an error may or may not be reported, and the error may be different each time. This obviously creates confusion for developers, and even I have stumbled here many times.

So, my advice is: Please turn off this feature. You can turn it off with the following two lines of code:

no_shuffle();
run_tests;

Among them, no_shufflethe primitive is used to disable randomness, so that the test runs strictly according to the order of the test cases.

reindex

Finally, let's talk about a less brain-burning and lighter topic. The test case set of OpenResty has strict requirements on the format. Each test case needs to be separated by 3 newlines, and the number of the test case must also be strictly self-increasing.

Fortunately, we have a corresponding automation tool reindexto do these tedious things, it is hidden in the [openresty-devel-utils] project, because there is no document to introduce it, few people know it.

Interested students can try to scramble the number of the test case, or add or delete the number of line breaks, and then use this tool to sort it out to see if it can be restored.

write at the end

test::nginxThis concludes the introduction to . Of course, there are actually more functions in it, and we only talked about the most core and important ones. It is better to teach a man to fish than to teach him to fish. I have already taught you the basic methods and points of attention for learning and testing, and you need to go to the official test cases to dig out the rest.

One last question for you. In your project development, are there tests? What framework are you using to test? Welcome to leave a message to discuss this issue with me, and you are welcome to share this article with more people to communicate and learn together.