Table of contents
Commonly used test pile mock capabilities in test work
http.server extension: implement a static file server with one line of command
Performance Optimization: Using Asynchronous Responses
Can be optimized: take advantage of multi-core
Start the service with gunicorn
Performance optimization: use cache (functools.lru_cache).
Commonly used test pile mock capabilities in test work
During our testing work, we may encounter scenarios where the development of the front-end service is completed and the dependent service is still under development; or we need to stress test a certain service, and the dependent components of this service (such as the test environment) cannot support concurrent access MQscenarios . At this time, we may need a service to replace these dependent components or services of the test environment, and this is the protagonist of this article - the test stub .
A test stub can be understood as a proxy that can be used to simulate external dependencies in the application, such as databases, network services or other APIs, which can help us isolate different parts of the application during development and testing, so that the test More reliable and repeatable.
Application Scenario
Test stubs are generally used in the following scenarios:
| Scenes | Reasons and purposes for using test stubs |
|---|---|
| unit test | Isolate the interaction of the code under test with other components or external dependencies, so that the code under test can be tested without considering other parts. |
| Integration Testing | When certain components are not implemented or are not available, use test stubs to simulate these components so that integration testing can continue. |
| Performance Testing | Quickly generate high load and a large number of concurrent requests, and evaluate the performance of the system under high load conditions. |
| Fault injection and recovery testing | Simulate failures (such as network failures, service downtime, etc.) to verify system behavior and resilience when encountering failures. |
| API testing | Use test stubs to simulate the response of the API so that client development and testing can take place before the API implementation is complete. |
| Third-party service testing | Avoid interaction with real third-party services during the development and testing phase, reducing additional costs and unstable test results. Test stubs are used to simulate these third-party services, enabling testing without affecting real services. |
This article will select several commonly used scenarios to introduce the development and optimization of test stubs step by step.
simple test stub
If it is inconvenient to install other libraries in the test environment, we can use a module http.servermodule in the Python standard library to create a simple HTTP request test stub.
# simple_stub.py
# 测试桩接收GET请求并返回JSON数据。
import json
from http.server import BaseHTTPRequestHandler, HTTPServer
class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
content = json.dumps({"message": "Hello, this is a test stub!"}).encode("utf-8")
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", f"{len(content)}")
self.end_headers()
self.wfile.write(content)
if __name__ == "__main__":
server_address = ("", 8000)
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
print("Test stub is running on port 8000")
httpd.serve_forever()
Run the above code, you will see that the test stub is listening on port 8000. You can use a browser or curlcommand to access http://localhost:8000, and you will receive {'message': 'Hello, this is a test stub!'}a response.
http.serverExtension: implement a static file server with one line of command
http.serverThe module can be used as a simple static file server for developing and testing static websites locally. To start the static file server, run the following command at the command line:
python3 -m http.server [port]
where [port] is an optional port number, which defaults to 8000 when not passed. The server will serve static files in the current directory.
If executed in the log folder python -m http.server, the files in this folder and the contents of subfolders can be accessed in a web browser:
Note:
http.serverMainly used for development and testing, performance and security do not meet the conditions for deployment in production environments
Performance Optimization: Using Asynchronous Responses
We implemented a simple test stub above, but in actual applications, we may need higher performance and more complex functions.
asynchronous response
In the case of only the same resources, such a service with network I/O, using an asynchronous method can undoubtedly make more effective use of system resources.
When it comes to asynchronous http frameworks, the most popular one is that it only takes two steps FastAPIto FastAPIimplement the above functions.
First, install FastAPI and Uvicorn:
pip install fastapi uvicorn
Next, create a fastapi_stub.pyfile called , with the following content:
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
async def get_request():
return {"message": "Hello, this is an optimized test stub!"}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000)
Execute the code, and this test stub is also listening on port 8000. We can use a browser or other HTTP client to initiate a request to the test stub as before.
Click to view the introduction to the advantages of asynchronous programming
Can be optimized: take advantage of multi-core
Although we used an asynchronous method to improve the performance of the test stub, the code still only runs on one CPU core. If we want to perform performance stress testing, it may not be able to meet our performance requirements. At this time, we can use gunicornthe library to take advantage of the multi-core advantages of the server.
gunicorn
Key features and benefits of Gunicorn:
| Features and advantages | illustrate |
|---|---|
| easy to use | Gunicorn is easy to install and configure, and can seamlessly integrate with many Python web frameworks (such as Flask, Django, FastAPI, etc.). |
| multi-Progress | Gunicorn uses a pre-forked working mode, creating multiple child processes to handle concurrent requests. This helps improve the performance and responsiveness of the application. |
| compatibility | Gunicorn follows the WSGI specification, which means it can be used with any Python web application that follows the WSGI specification. |
| configurability | Gunicorn provides many configuration options such as number of worker processes, worker process type (sync, async), timeout settings, etc. This allows Gunicorn to be flexibly configured according to specific needs. |
| deployment friendly | Gunicorn is very popular in production environments because it simplifies the deployment process. Gunicorn can be used with other tools such as Nginx, Supervisor, etc. to better manage and scale web applications. |
install gunicorn
pip install gunicorn
Start the service with gunicorn
Start the service:
gunicorn -w 4 fastapi_stub:app
As you can see, the above command starts 4 worker processes, and you can also use ps -efthe command to query the process status.
Some common parameters of gunicorn:
| parameter | illustrate |
|---|---|
| -w, --workers | Set the number of worker processes. Adjust according to the number of CPU cores of the system and the load characteristics of the application. The default value is 1. |
| -k, --worker-class | Set the type of worker process. Can be sync(default), gevent, , eventletetc. If you use asynchronous worker processes, you need to install the corresponding library. For example, for FastAPI applications, you can use -k uvicorn.workers.UvicornWorker. |
| -b, --bind | Set the address and port the server binds to. The format is address:port. For example: -b 0.0.0.0:8000. The default value is 127.0.0.1:8000. |
| --timeout | Set the timeout in seconds for worker processes. If the worker process does not complete the task within the specified time, it will be restarted. The default is 30 seconds. |
| --log-level | Set the log level. Can be debug, info, warning, erroror critical. The default value is info. |
| --access-logfile | Set the path to the access log file. By default, access logs will be output to the standard error stream. To disable access logging, use -. For example: --access-logfile -. |
| --error-logfile | Sets the path to the error log file. By default, error logs will be output to the standard error stream. To disable error logging, use -. For example: --error-logfile -. |
| --reload | Use this option in a development environment and Gunicorn will automatically reload when the application code changes. Not recommended for use in production environments. |
| --daemon | Use this option to run Gunicorn in daemon mode. In this mode, Gunicorn will run in the background and detach automatically on startup. |
Gunicorn offers many other configuration options that can be tuned to specific needs. For a complete list of options, you can check the official Gunicorn documentation: https://docs.gunicorn.org/en/stable/settings.html.
Performance optimization: use cache( functools.lru_cache).
When dealing with repetitive computation or data retrieval tasks. Using memory cache (such as Python's functools.lru_cache) or external cache (such as Redis) to cache frequently used data can also greatly improve the efficiency of test stubs.
Assuming that our test pile needs to use time-consuming functions such as calculating the Fibonacci sequence, then caching the results and returning them directly when the same request is encountered next time instead of calculating and then returning will greatly improve the use of resources rate, reducing the waiting time for a response .
If it just returns the data directly, and there is no test stub for complex calculations, it
lru_cachehas no practical significance to use.
Here's an example of a more appropriate usage lru_cache, where we'll perform calculations on the Fibonacci sequence and cache the results:
from fastapi import FastAPI
from functools import lru_cache
app = FastAPI()
@lru_cache(maxsize=100)
def fibonacci(n: int):
if n <= 1:
return n
else:
return fibonacci(n - 1) + fibonacci(n - 2)
@app.get("/fibonacci/{n}")
async def get_fibonacci(n: int):
result = fibonacci(n)
return {"result": result}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000)
In this example, we create a simple HTTP request test stub using FastAPI. We define a fibonaccifunction named , which calculates the Fibonacci sequence. To improve performance, we functools.lru_cachecache this function.
In the route /fibonacci/{n}, we call fibonaccithe function and return the result. Command access is available http://localhost:8000/fibonacci/{n}for debugging.
It should be noted that maxsizethe parameter is functools.lru_cachea configuration option of the decorator, which represents the maximum capacity of the cache. lru_cacheUsing a dictionary to store cache entries, when a new result needs to be cached, it checks the current cache size. If the cache is full (i.e. reached maxsize), the least recently used cache entry is removed according to the LRU policy. If maxsizeset to None, the cache can grow without bound, which can cause memory issues.
mock in unit test
Python unittest.mock
In Python, the unittest module provides a submodule called unittest.mock for creating mock objects. unittest.mock contains a class called Mock and a context manager/decorator called patch that can be used to replace dependencies in the code under test.
import requests
from unittest import TestCase
from unittest.mock import patch
# 定义一个函数 get_user_name，它使用 requests.get 发起 HTTP 请求以获取用户名称
def get_user_name(user_id):
response = requests.get(f"https://api.example.com/users/{user_id}")
return response.json()["name"]
# 创建一个名为 TestGetUserName 的测试类，它继承自 unittest.TestCase
class TestGetUserName(TestCase):
# 使用 unittest.mock.patch 装饰器替换 requests.get 函数
@patch("requests.get")
# 定义一个名为 test_get_user_name 的测试方法，它接受一个名为 mock_get 的参数
def test_get_user_name(self, mock_get):
# 配置 mock_get 的返回值，使其在调用 json 方法时返回一个包含 "name": "Alice" 的字典
mock_get.return_value.json.return_value = {"name": "Alice"}
# 调用 get_user_name 函数，并传入 user_id 参数
user_name = get_user_name(1)
# 使用 unittest.TestCase 的 assertEqual 方法检查 get_user_name 的返回值是否等于 "Alice"
self.assertEqual(user_name, "Alice")
# 使用 unittest.mock.Mock 的 assert_called_with 方法检查 mock_get 是否被正确调用
mock_get.assert_called_with("https://api.example.com/users/1")
Summarize
When developing a test stub, we need to design the behavior of the test stub according to the actual needs and the characteristics of the back-end service, in order to make it closer to the behavior of the actual back-end service and ensure that the test results have higher reliability and accuracy .
There may be other optimization solutions, and suggestions are welcome. I hope this article can help everyone in their work.
If you think it's not bad, just like it in the lower right corner, thank you!
Data acquisition method
【Message 777】
Friends who want to get source code and other tutorial materials, please like + comment + bookmark , triple!
After three times in a row , I will send you private messages one by one in the comment area~