Read this article to get started - game testing

All good books read like conversations with some of the world's greatest minds from the past.

I was fortunate to see a game testing course taught by a teacher, and I benefited a lot. Combined with my current work, I compiled this blog. After understanding game testing, it will really expand one's thinking. Although this blog mainly shares the tests of game testing, it is also applicable to tests for other industries such as live broadcasting. This article is suitable for beginners who are new to the software testing industry.

1. Introduction to the game development team

1.1 Producer - the overall person in charge of the project

负责游戏研发环节
负责游戏运营环节
负责项目人员管理
负责项目事物管理

1.2 Planning - Designers

剧情策划
	负责规划游戏中的各种剧情、故事、背景等。
系统策划
	设计游戏中各种系统的规则
数值策划
	规划游戏中各种资源的产出、消耗等
关卡策划
	设计游戏中各种关卡

1.3 Development - code implementers

负责把策划的设计及美术资源等通过编码实现可见的程序。开发也分为后端开发人员和前端开发人员
后端开发：实现服务器端的逻辑、数据验证等
前端开发：实现游戏客户端端展现与逻辑

1.4 Art

制作游戏中的各类美术资源
包括场景、原画、UI、动画等

1.5 Test

项目的质量保证人员，主要工作是发现游戏中存在的缺陷并及时反馈出来
测试也可分为功能测试、性能测试、压力测试、兼容性测试、自动化测试、安全测试

2. Game development process

1. Producer: specify project goals and plan goals
2. Planning: Disassemble the project goals into detailed requirements, and refine the requirements into documents
3. Testing, development, and art: implement requirements with code and art resources, test and write test cases
4. Tester: Perform quality control on all aspects of the project, and feedback the defects found.

3. Main content of game testing

Test content for other industries, also covered

3.1 Functional test

• It is the most common test mode, and the main test method is black box test
• Mainly used to verify whether the function meets the requirement design
• Mainly consider the correctness of the function, regardless of the underlying structure of the game and code errors
• Usually start testing from the interface, trying to simulate possible user operations

3.2 Client performance test

Some indicators of performance testing

• Client CPU usage
• Client memory usage
• Client Network Traffic Usage
• Client Power Consumption
• Client frame rate (FPS)
• The instrument that comes with Xcode, a common tool for ios
• Android common tools emmage and GT

3.3 Stress test

• Server CPU usage
• Server memory usage
• System Throughput (TPS)
• transaction response time
• transaction success rate

3.4 Compatibility Test

• Model Adaptation Test
• OS Compatibility Testing
• Screen Resolution Compatibility Test
• Game Version Compatibility Test

Supplement: Whether it is a compatibility test or an ordinary iterative test, we require model coverage. Generally speaking, we will choose mid-to-high-low models for testing, and the judgment of mid-to-high-low models is mainly through the processor. Decided to recommend a mobile phone processor ladder map to everyone

3.5 Security Testing

• memory modification test
• Client Encryption Test
• Client decompilation test
• Network Security Test

3.6 Interface Test

• The data test of each interface of the server is mainly realized through tools (tool: jmter)
• Interface security test, send request repeatedly, check interface processing status

3.7 Log Test

• client log
• Server logs
  Through some logs of the client and server, it is possible to simulate the customer's usage posture, or to simulate the player's behavior.

3.8 Weak network test

• The running status of the game under different network conditions, such as edge, 2G, 3G, 4G, etc.
• How the game works under different packet loss rates
• When switching between different network signals, the impact on product operation
• The impact of network disconnection and reconnection on product operation
• Issues related to the consistency of front-end and back-end data.

3.8.1 Weak network tools

• Mac systems can use Network Link conditionerorCharles
• Windows systems can use Network Emulator for Windows Toolkit,Fiddler

For the weak network test, you can refer to another article of the editor.

3.9 gm tool test

Commonly used tools in the game

• To test the function implementation of the gm tool, you need to pay attention to whether the tool settings work in the game
• Test the data reading and storage of the gm tool

3.10 SDK testing

• user data test
• Recharge, consumption test
• Docking test with various channels

4 Basic process of game testing

Although this blog mainly shares the tests of game testing, it is also applicable to tests for other industries such as live broadcasting.

4.1 Functional meeting

• Understand the content of functional requirements
• Point out possible risks
• Think about the key points and difficulties of the test function, if you need tool assistance, you need to put forward the development requirements
• Think about where features can be optimized and bring up discussions

4.2 Writing test cases

• Write test cases according to requirements
• Focus on functional logic implementation
• Consider various special cases such as boundary values, network outages, process outages, etc.
• Pay attention to changes in requirements, as requirements change frequently, and test cases need to be adjusted in time

4.3 Smoke test

• A link before detailed testing to quickly find obvious bugs
• Quickly ensure that the main logic process runs smoothly
• Quickly clarify the status of function development

4.4 Detailed tests

• Meticulous testing of logical branches, resources, and configurations
• Try to simulate every possible operation of the user
• Test abnormal conditions, such as network disconnection, power failure, event interruption, process interruption, etc.
• Test data reading, storage, network, etc.
• Test the effect of this function on other functions

4.5 Regression testing

• Tests have been fixed etc.
• Adjusted content of test requirements
• Test each logical branch in detail again

4.6 Checklist inspection (non-essential test)

• The main logic point of the quick check function
• Check for any other function points that are associated with this function

5 Notes on writing test cases

5.1 Analysis of requirement documents

• Document reading
  First of all, we need to analyze the requirements by file, find the person in charge for detailed communication, logical sorting, and problems related to function expansion and compatibility. For compatibility, you can ask about the support status of the lowest and highest models. The requirements document should be read at least three times, which can deepen the understanding of the function.
• Detailed communication.
  If you don’t understand something, you should confirm it in time, and don’t take it for granted. If there is a change in the requirements document, be sure to double check with the programmer and the tester.
• Logical combing
  Although we rarely write code for testing, we also need to understand the logic of function implementation. Some requirements documents are not necessarily written according to the process, and there may be places where functions overlap, so we need to sort out the framework and gradually refine it. .
• Thinking about function expansion
  You can see if there are any defects in the function design idea, think about the difficulty of testing, think about the degree of relevance, and think about special circumstances.
• Compatibility-related thinking
  version compatibility, function compatibility, operating system version compatibility, resolution compatibility

5.2 Functional module division

The principle of functional division: 高内聚低耦合、重整体轻局部. It is probably based on the internal functions and between modules.
Different division methods are applicable to different scenarios, which should be analyzed according to specific problems

5.2.1 Functional process division method

Draw the basic process of the function, divide the modules according to each major link of the process, and then refine and check for gaps

5.2.2 Hierarchy division method

The process of refining the modules layer by layer according to the logical level is more suitable for UI division and relatively large system module division.

5.2.3 Type division method

Divided according to the different types of functional content

6 Test case writing

When writing test cases, we need to pay attention to the format. For the format, you can choose the form or flow chart, whichever you are familiar with, and the most important thing is that it can cover all functions, which can be seen at a glance. Test cases need to contain the following:

• use case name
• Version
• Writer, date of writing, remarks (writer and date of writing must be written, others are inconspicuous, and when it is useful)
• The link or address of the requirements document (information related to this test must be put up, which can be used as background information and test source)
• module information
• Input information, output results
• Remarks

Supplement:
We must have heard of the commonly used methods of writing test cases: 等价类, 边界值, 因果图和判定表.

• 等价类
  It refers to that in an input set, any input data is equivalent to the verification of the output. At this time, a small amount of representative test data can be selected to represent the overall data. Equivalence classes are further divided into valid equivalence classes and invalid equivalence classes.
  Example of equivalence class: If you buy a product and the price of the product is 12 yuan, then we need to take 11, 12, 13 3 data to test and verify, which can represent all data tests, thus saving a lot of time.
• 边界值
  A method of analyzing the boundary values ​​of input and output. The determination of boundary values ​​generally selects exactly equal to, just less than and just greater than, usually applicable categories: numerical test, string test, data type test and other boundary
  values Example: range 2000-3000, then we need to test the boundary value, the test value: 1999, 2000, 2001, 2999, 3000, 3001
• 因果图和判定表
  The two are closely related. The judgment table is generated through the causal diagram, and the test cases are written through the judgment table. A causal diagram is a relationship diagram of the causal relationship between input and output.
  Decision table: A result judgment table can be generated through a causal diagram.

6.1 Notes on writing test cases:

• The input conditions should be single and clear, and try not to use words that are easy to misunderstand, such as possible, probably, etc.
• The output should be determinable and unambiguous. It is best not to use the term "display correct".
• The test steps should be executable
• Keep coverage as high as possible
• Abstract as much as possible to avoid meaningless redundancy

7 Test case organization and maintenance

• After the requirements change, it is necessary to update the old test cases in time, and write a clear note about the modification
• Test cases should try to avoid redundancy. If you encounter repeated use cases, you need to modify them according to the actual situation
• Pay attention to the backup of test cases. After writing, it is best to back up a copy locally to avoid being accidentally deleted online.

8 Bug Definition Criteria

• Does not match the design requirements
• against common sense

8.1 Bug classification

• P0: Fatal error, which needs to be repaired immediately, such as downtime, restart error reporting, etc.
• P1: Serious errors that require urgent repairs, such as functional process errors, numerical errors, etc.
• P2: General errors, allowing for a period of time to fix, such as simple errors such as functions, interface errors, etc.
• P3: Irrelevant errors, which allow deferred repairs, such as text errors, missing pixels, etc.

8.2 What to include in a bug report

It doesn't matter if some companies use Jira and others use emails or forms, the important thing is to include the following information.

• Title: module name + short description
• Test environment: indicate the version used for testing, system, server, account, etc.
• bug detailed description
• Steps to reproduce: the detailed process steps and probability of reproducing the bug, etc.
• Expected result: hope the result after the bug is fixed
• Attachments: logs, screenshots, videos, etc.

8.3 Bug Acceptance Criteria

• Expansion: whether it has an impact on other functions, a simple regression is required
• Note: Verification should not only focus on the front-end display, but also focus on
  the tracking and promotion of back-end data bugs
• Everyone is responsible for tracking the fix status of their own bugs
• Communicate with developers in a timely manner to understand the repair status and provide support during the repair process
• Bugs that have not been fixed for a long time need to be confirmed with the development and superiors on how to deal with them
• After the bug is fixed, it needs to be verified in time

8.4 Bug analysis

We can analyze the bug from the following latitudes

• Analysis according to Bug level: list P0, P1, P2, P3 types of bugs for analysis
• According to the bug data of the project personnel, there are two methods. The first method is to analyze the number of bugs detected by the testers, and the second method is to analyze the number of bugs assigned to the development.
• According to the function module Bug data analysis