If you want to develop ClickHouse, you must not only know how to write code, but also know how to do a good job of testing, so as to ensure that the functions you develop are available. So how to write test cases is very important. This article introduces how ClickHouse writes test cases.
There are many types of tests in ClickHouse. Here we mainly introduce the two most commonly used tests, namely stateless tests and integration tests.
stateless testing
This is a functional test, which includes stateless tests and stateful tests, namely stateless tests and stateful tests. Most functional development can be verified by writing functional tests. A stateless test refers to running a query without preloading any test data. You can build tables, write data, and perform various SQL operations in the test to verify functions. Stateful testing needs to preload test data from ClickHouse, which is generally rarely used, so I won’t introduce it here.
Now use an example to illustrate how to write stateless test cases.
The stateless test cases are all in ClickHouse/tests/queries/0_statelessthe directory, to add test cases also need to be added in this directory. For example, we want to create a MergeTree table, then write a piece of data, and verify that the data is written successfully. Examples are as follows:
Create a SQL file such as 00000_check_mergetree.sql:
select 'check MergeTree insert 1';
create table if not exists 00000_check_mergetree (id Int32) engine = MergeTree() order by id;
insert into 00000_check_mergetree values (1);
select id from 00000_check_mergetree;
drop table 00000_check_mergetree;
Here is an explanation of why 00000_check_mergetree is used as the name of the table. It is because stateless test cases will use a server to test, and making the table name and folder name consistent can avoid conflicts with other use cases.
Then execute the following command in the ClickHouse directory:
./build/programs/clickhouse-client --port 9000 --multiquery < tests/queries/0_stateless/00000_check_mergetree.sql > tests/queries/0_stateless/00000_check_mergetree.reference
Where ./build/programs/clickhouse-clientis the path of our compiled clickhouse-client binary. For how to compile ClickHouse, please refer to the simplest method of building a compilation environment for ClickHouse . ClickHouse needs to be started before executing the above statement, because the above command needs to actually access ClickHouse. After the above statement is executed, tests/queries/0_stateless/00000_check_mergetree.referencethe execution result will be generated in the file we specified.
The execution results are as follows:
1
Note here that a blank line will be generated at the end, so when writing a test case, you must use the above statement to generate .referencea file instead of writing it by hand.
So far, a simple test case has been written.
In fact, some people will have some doubts about this. .referenceThere is only one 1 in the file. I don’t understand what it means. If you want to add an explanation, how do you add it?
We can 00000_check_mergetree.sqlmodify the file:
select 'check MergeTree insert 1';
create table if not exists 00000_check_mergetree (id Int32) engine = MergeTree() order by id;
insert into 00000_check_mergetree values (1);
select id from 00000_check_mergetree;
drop table 00000_check_mergetree;
The final 00000_check_mergetree.referencefile is:
check MergeTree insert 1
1
Some people may think that this is so low, don’t worry, ClickHouse provides a more elegant way, let’s rewrite the test case below:
-- {echo}
create table if not exists 00000_check_mergetree (id Int32) engine = MergeTree() order by id;
insert into 00000_check_mergetree values (1);
select id from 00000_check_mergetree;
drop table 00000_check_mergetree;
-- {echo}Both the SQL statement and the result can be output through , the result is as follows:
-- {echo}
create table if not exists 00000_check_mergetree (id Int32) engine = MergeTree() order by id;
insert into 00000_check_mergetree values (1);
select id from 00000_check_mergetree;
1
drop table 00000_check_mergetree;
In this way, it is easy for us to compare the test statement and the execution result.
In addition, stateless tests also support adding Tags. For example, when we need to create ReplicatedMergeTree and need to rely on zk, then we need to add zookeeper tags, for example:
-- Tags: zookeeper
In addition, ClickHouse also supports many tags, please refer to the official website: test tags .
integration test
This is an integration test, and sometimes the functional test cannot meet the needs, such as the need to simulate a test scenario of a cluster with 2 shards and 2 copies, or simulate a scenario where a node in the cluster cannot be connected.
Test cases such as integration tests are in ClickHouse/tests/integrationthe directory. Generally, the directory structure of a complete test case is as follows:
.
├── configs
│ └── servers.xml
├── __init__.py
└── test.py
Similarly, we add a use case under this directory to simulate creating a cluster with 2 shards and 2 copies, and query the system.clusters table. First create a directory test_cluster, and create an empty file according to the above directory structure.
Configure the cluster topology in servers.xml:
<clickhouse>
<remote_servers>
<two_shards_two_replicas>
<shard>
<replica>
<host>node1</host>
<port>9000</port>
</replica>
<replica>
<host>node2</host>
<port>9000</port>
</replica>
</shard>
<shard>
<replica>
<host>node3</host>
<port>9000</port>
</replica>
<replica>
<host>node4</host>
<port>9000</port>
</replica>
</shard>
</two_shards>
</remote_servers>
</clickhouse>
Write the test method in test.py:
import pytest
from helpers.cluster import ClickHouseCluster
cluster = ClickHouseCluster(__file__)
node1 = cluster.add_instance(
"node1", main_configs=["configs/servers.xml"], with_zookeeper=True
)
node2 = cluster.add_instance(
"node2", main_configs=["configs/servers.xml"], with_zookeeper=True
)
node3 = cluster.add_instance(
"node3", main_configs=["configs/servers.xml"], with_zookeeper=True
)
node4 = cluster.add_instance(
"node4", main_configs=["configs/servers.xml"], with_zookeeper=True
)
@pytest.fixture(scope="module")
def start_cluster():
try:
cluster.start()
yield cluster
finally:
cluster.shutdown()
def test_system_cluster(start_cluster):
node1.query("select * from system.clusters;")
After writing everything, enter ClickHouse/tests/integrationthe directory and execute the following statement:
./runner --binary /data/ClickHouse/build/programs/clickhouse --base-configs-dir /data/ClickHouse/build/programs --command bash
Among them, /data/ClickHouse/build/programs/clickhouseis the path of the compiled clickhouse binary, /data/ClickHouse/build/programsand the path of the basic configuration file of ClickHouse, so a config.xml and a users.xml file need to exist under this path. For details, refer to the configuration file in ClickHouse Quick Build .
After executing the above command, a docker will be started, which contains all the environment dependencies required by the integration test. However, due to the limitations of docker download images, you need to use docker login to log in to your own docker account, because the integration tests are all tested by starting the node through docker. Next, we can execute the test command to run our test. The command is as follows:
pytest -ss test_cluster/test.py -vv
The normal output is as follows:
========================================================== test session starts ==========================================================
platform linux -- Python 3.8.10, pytest-7.2.0, pluggy-1.0.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /ClickHouse/tests/integration, configfile: pytest.ini
plugins: order-1.0.0, repeat-0.9.1, timeout-2.1.0, xdist-3.0.2
timeout: 900.0s
timeout method: signal
timeout func_only: False
collected 1 item
test_cluster/test.py::test_system_cluster Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
PASSED
========================================================== 1 passed in 12.36s ===========================================================
If the output is an error like:
FAILED test_cluster/test.py::test_system_cluster - helpers.client.QueryRuntimeException: Client failed! Return code: 210, stderr: Code: 210. DB::NetException: Connection reset by peer...
You need to change the iptables settings of the host machine and execute:
sudo iptables -P FORWARD ACCEPT
If you want to test an abnormal situation, you can change the test_system_cluster in test.py to the following form:
def test_system_cluster(start_cluster):
node1.query("select * from system.clusters;")
error = node1.query_and_get_error("select * from system.clustersxxx;")
assert "UNKNOWN_TABLE" in error
If you want to test a normal situation, you can modify it like this:
def test_system_cluster(start_cluster):
assert (
node1.query("select cluster from system.clusters where cluster = 'two_shards_two_replicas';")
== "two_shards_two_replicas\ntwo_shards_two_replicas\ntwo_shards_two_replicas\ntwo_shards_two_replicas\n"
)
error = node1.query_and_get_error("select * from system.clustersxxx;")
assert "UNKNOWN_TABLE" in error
Therefore, you can select the corresponding verification method according to actual needs.
So far, the two main testing methods have been introduced.
Welcome to add WeChat: xiedeyantu to discuss technical issues.