AREX is an open source automated regression testing platform based on real requests and data (project address: https://github.com/arextest), using Java Agent technology and comparison technology to achieve fast and effective regression through traffic recording and playback capabilities test. At the same time, it provides a wealth of automated testing functions such as interface testing and interface comparison testing, so you can get started quickly without programming skills.

When using the AREX traffic recording function, the AREX Java Agent will record the data traffic and request information of the Java application in the production environment, and send this information to the AREX data access service (Storage Service), which will be imported into the Mongodb database to store. When replay testing is required, the AREX Schedule Service will extract the recorded data (request) of the application under test from the database through the data access service according to the user's configuration and needs, and then send an interface request to the target verification service . At the same time, the Java Agent will return the recorded external dependency (external request/DB) response to the application under test, and the target service will return a response message after processing the request logic. Then the scheduling service will compare the recorded response message with the playback response message to verify the correctness of the system logic, and push the comparison result to the analysis service (Report Service), which will generate a playback report for testers to check . Throughout the process, AREX's cache service Redis is responsible for caching the Mock data and comparison results during playback to improve comparison efficiency.

This article takes community-testthe tool as an example to fully demonstrate the whole process of automated regression testing of the AREX interface, from AREX Agent configuration to recording and playback, and problem location.

community-test( https://github.com/arextest/arex-community-test) is a tool provided by AREX for testing and verifying AREX system functions. It is a Java application that relies on MySQL and Redis for simulation business services.

AREX operating environment

The author applied for a Linux server in the application environment, which is a device provided by OPS.

The IP address of the server is 10.5.153.1, it has 128GB memory and 32 Cores. The server can be accessed through the company's office network, but cannot be accessed by other external environments. The main purpose is to install AREX service and deploy community-test test environment on this server.

In order to ensure that the server can access the public network, some settings need to be made. During the installation process, you need to download the installation package of AREX, and obtain the code of community-test to compile and release. If your server can directly access the public network, you can skip the step of setting up a proxy. Otherwise, a proxy needs to be set to ensure that the server can access the public network.

In terms of ports, there is no limit to the number of ports exposed to the outside world. However, OPS now restricts each server. By default, only ports 80 and 8080 for external access are allowed, and other ports are blocked by default.

In addition, remote access is limited, and this demonstration is to log in to the server through a springboard machine.

Regression testing by recording playback

Install AREX

cd arex
git clone https://github.com/arextest/deployments.git

Execute the following command to start AREX:

# 读取当前目录的docker-compose.yml文件,并以服务的方式启动
docker-compose up -d
# 读取当前目录的docker-compose-distribute.yml文件,并以服务的方式启动
docker-compose -f docker-compose-distribute.yml up -d
# 读取当前目录的docker-compose-mongo4.4.21.yml文件,并以服务的方式启动, 因为某些服务器硬件版本的原因,不能启动5.0的mongodb
docker-compose -f docker-compose-mongo4.4.21.yml up -d

After startup, you can use docker-compose psthe command to view the status and configuration of each service:

State is the state of each service, and Up is correct. If it is in other states, you need to restart the service or check the service log to check why it cannot be started.
arex-front is the front end, here my port has not been modified, the default is port 8088. You can configure the desired port according to your environment, such as port 80.
The port of MongoDB is still the default 27017 (you can also use your MongoDB tool to connect to the database), the link address is: 10.5.153.1:27017, the user name is "arex", and the password is "iLoveArex". Please note that in the configuration of Docker Compose, the link address should be: mongodb://arex:iLoveArex @mongodb :27017/arex_storage_db, because the service name is used for network connection in Docker Compose. Therefore, you can access MongoDB using the following address: mongodb://arex:iLove [email protected] :27017/arex_storage_db.
The port of arex-storage is 8093, which is the port that needs to be specified in the agent configuration. The storage address configured in my environment is 10.5.153.1:8093.

Use docker-compose imagesto view the running version of each service component:

Each component of AREX is 0.2.10, and arex-nodethe service will be removed later, and the function has not been updated, it is still version 0.2.7.

After startup, there are two directories under the current startup directory. Among them arex-logsis the log of each service, arex-datawhich is used to store data.

After startup, you can view the log on the command line:

docker-compose logs: view all logs
docker-compse logs arex-report-service: View the logs of the Report service

After installation, visit the AREX front-end page http://10.5.153.1:8088/, enter the email address to obtain the verification code to log in, as shown below:

So far, the installation of AREX is over.

community-test business service installation

Download and compile AREX Agent code

git clone https://github.com/arextest/arex-agent-java.git
cd arex-agent-java
mvn clean package

Compilation is complete:

Check the arex-agent-jar directory in the current directory to see if there is the latest compiled arex agent jar file:

Compile the community-test code

Pull code:

git clone https://github.com/arextest/arex-community-test.git

mvn clean packageCompile the project with

For the convenience of operation, containerized deployment will community-testbe performed :

FROM tomcat:9.0-jdk8-openjdk

ARG WAR_FILE=./target/arex-agent-test-0.0.1-SNAPSHOT.war
ADD $WAR_FILE /usr/local/tomcat/webapps/

WORKDIR /usr/local/tomcat/conf
RUN sed -i 'N;152a\\t<Context path="" docBase="arex-agent-test-0.0.1-SNAPSHOT" reloadable="true" />' server.xml

ADD ./arex-agent-0.2.0.jar /usr/local/tomcat/
ADD ./arex-agent-bootstrap-0.2.0.jar /usr/local/tomcat/

WORKDIR /usr/local/tomcat
EXPOSE 8080
CMD ["catalina.sh","run"]

The containerized compilation shell is as follows, for reference only, and the relative directory may need to be modified:

cd ../arex-community-test
mvn clean package
 
cp ../deployments/dockerfile/community.Dockerfile ./Dockerfile
cp ../arex-agent-java/arex-agent-jar/arex-agent-0.3.0.jar ./arex-agent-0.3.0.jar
cp ../arex-agent-java/arex-agent-jar/arex-agent-bootstrap-0.3.0.jar ./arex-agent-bootstrap-0.3.0.jar
 
docker build -t arexadmin01/arex-community-test:0.0.1  .
 
rm -rf ./Dockerfile
rm -rf ./arex-agent-0.3.0.jar
rm -rf ./arex-agent-bootstrap-0.3.0.jar
 
cd ..

So far, the program image for testing has been successfully created.

SUT application configuration AREX Agent

The following is the original command line used when running the community-test tool, and provides the connection configuration with the database and Redis, as follows:

environment:
      - JAVA_TOOL_OPTIONS=-Dspring.datasource.url=jdbc:mysql://cmysql:3306/community?useUnicode=true&characterEncoding=utf-8 -Dspring.datasource.username=arex_admin -Dspring.datasource.password=arex_admin_password -Dspring.redis.host=credis -Dspring.redis.port=6379

Add AREX configuration:

environment:
      - JAVA_TOOL_OPTIONS='-javaagent:/usr/local/tomcat/arex-agent-0.3.0.jar' -Darex.service.name=community-service -Darex.storage.service.host=10.5.153.1:8093 -Darex.enable.debug=true  -Dspring.datasource.url=jdbc:mysql://cmysql:3306/community?useUnicode=true&characterEncoding=utf-8 -Dspring.datasource.username=arex_admin -Dspring.datasource.password=arex_admin_password -Dspring.redis.host=credis -Dspring.redis.port=6379

'-javaagent:/usr/local/tomcat/arex-agent-0.3.0.jar' This part is the JAR file of the AREX Agent we compiled, and the JAR file will be loaded into the application as a Java agent (Java agent).
-Darex.service.name=community-service: This is the name of the application, which will be displayed on the AREX Replay page.
-Darex.storage.service.host=10.5.153.1:8093: This is the address of the AREX Storage service, specified as 10.5.153.1:8093.
-Darex.enable.debug=true: This is a configuration option, if set to true, it will run in debug mode, that is, all traffic will be recorded. In a production environment, it is recommended to set this to false.

Start arex-community-testthe service , and then check the application registration status on the AREX front-end page:

It can be seen that the tested application community-testhas successfully run, and its access address is http://10.5.153.1:8080/.

Regression Testing

Production environment release and run

Now, the arex-agent has been successfully started and running in the application. During the recording process, the user does not need to perform any special operation or intervention, and can provide external services or manually access the application in the usual way.

In the demonstration of this article, the author uses the batch execution function in Collection community-serviceto access, and the access address is http://10.5.153.1:8080/.

During the access process, arex-agent will automatically start recording and store the recorded AREX use cases, without manual recording operation by the user.

Business code update

After our business requirements are realized and submitted, the new version needs to be compiled and tested, and the following steps are performed:

pull new code
Compile the code and package it
Important: Keep the configuration of AREX Agent unchanged, especially the application name, which is used by AREX to identify applications and manage use cases. Therefore, during the compilation and testing of the new version, the same application name configuration as the previous version should be maintained.
Release the code to the test environment: Release the compiled and packaged application code to the test environment for further testing and verification.

Regression testing in test environment

Enter the AREX Replay page, select community-serviceService , click the "Start Replay" button on the playback interface on the right, enter the playback address (I use the same environment, so the address is still http://10.5.153.1:8080/), and start playback :

Test problem location

The test passes if the test comparison is executed in full without any differences:

If problems are found during testing, as follows:

Click on the playback report with problems to display all interfaces and their regression tests, as shown in the figure below:

Clicking DiffScenes(New) will display the difference points in the overall statistics view. value diff , which is the value difference between the old and new versions.

Continue to click the difference point to view the difference details, as shown in the figure below. On the left is benchmark, which is the value recorded in production; on the right is Test, which is the value returned during the playback of the test environment. If the two are inconsistent, the difference information will be returned.

Based on the differences found, find the problematic point in the code:

Confirm the problem, fix the problem, repeat the "Business Code Update" chapter, modify→test release→compare.
If it is confirmed that it is not a problem, then set this node as a filter node, and skip this node comparison in the next playback.
Continue this operation to confirm that all differences are fixed or the differences are within the expected range.
Confirm fixes and releases.

AREX Documentation: http://arextest.com/zh-Hans/docs/intro/

AREX official website: http://arextest.com/

AREX GitHub：https://github.com/arextest

AREX official QQ communication group: 656108079

Super easy-to-use regression testing tool, I don't allow you to use it yet!