Software Testing - Advanced

Review the main content of the previous blog: use cases

How to design test cases according to requirements?

1. Verify the correctness, rationality, non-ambiguity, and self-consistent logic of the requirements
2. Analyze requirements, refine requirements, propose test items from requirements, find test points based on test items, and design test cases based on test points

What are the two aspects of designing test cases according to requirements?
functional and non-functional

1. Functional
   All the functions of the interface cannot be omitted (from top to bottom, left to right).
   Connect functions together to form scenarios or services, and test scenarios and services.
   Multiple inputs for a function must be
   input for abnormal data of the test function. Abnormal operation to test
2. Non-functional
   Reliability, usability, fault tolerance, compatibility, portability, performance, security, maintainability
   Different types of software, non-functional testing focuses on different
   large-scale commercial software (Taobao, WeChat , Weibo, etc.), software installed by the client, software used inside the B/S enterprise, Word, office (stand-alone installation)

What are the specific methods of designing test cases?
Equivalence classes, boundary value, scenario method, error guess method, causal diagram method, orthogonal method

1. According to the test object division

1.1 Interface Test

Users interact with the software through the interface, and the quality of the interface design directly affects the user's image of the software.

Interface testing (referred to as UI testing) refers to testing and checking all the content displayed on our software interface according to the interface requirements (usually UI design draft) and interface design rules, generally including the following:

1. Test the integrity, correctness and consistency of software interface elements.
2. The layout of the software interface is reasonable, with fonts and colors.
3. Test the adaptability of the interface, the interface adapts to different page sizes (the text does not overlap, disappear, the functions are all there, can be used normally, the pictures are clear and reasonable, mobile phones, iPads, computer screens, the interface changes from major to small (from small to large) In the process, the function is complete, the text is complete, the picture is complete, there will be no superimposition, disappearance, and the function cannot be used).
4. Interface controls are functional, dialog scrollbars, buttons. It is possible to distinguish between the active state and the inactive state of the button.
5. Interface design (colors, layout).

1.2 Reliability test

Reliability refers to the software's ability to function properly, the percentage of time the software is up and running overall.

Reliability = uptime / (uptime + non-uptime) * 100%

Factors affecting software reliability: Network, software environment (installation), hardware environment, software itself, abnormal environment will make the software run abnormally

Different software has different requirements for reliability. The reliability requirements for non-real-time software are generally 99.99%, 99.95% (mail system) and
special software has high reliability, 99.999% (military system)

If the availability reaches 99.99%, for a system that runs uninterrupted (7*24) throughout the year, it means that the time for the year (252600min) to not work normally is only 52min, less than an hour.
If the availability reaches 99.999%, it means that the non-working time is only 5 minutes throughout the year.

怎样测试软件的可靠性？
For a week, write down the time of failure to calculate the percentage

1.3 Fault tolerance test

When an exception occurs in the system, or an error occurs in the software system due to incorrect operation, the software digests the error by itself, or modifies it without letting the customer know, which is called the fault tolerance of the system.

1. Common fault-tolerant processing
   数据容错性：ATM input money less than 100 (prompt) Time: 25:70, year, month, day: February 30th, other months 32nd (prompt) before and after spaces (
   校验容错性：automatic filter); verify uppercase and lowercase letters (verification code: automatically converted to uppercase and lowercase), information verification before and after the same form or file (ID card, student number, automatic front and rear verification) tips for complex operations, tips for dangerous
   界面容错性：operations ;
   环境容错性：The environment where the shielding software of the dangerous button is located fails, and the software has a backup plan; it allows the user to switch without perception. (network, electricity, hardware environment, software environment for software deployment)
2. Disaster recovery test
   Artificially let the system fail, see if the system itself stores and restores user data quickly

1.4 Documentation test

Testing of relevant documentation for software development.

Documentation testing concerns:

1. Documentation Terminology
2. correctness of documentation
3. Document integrity
4. Documentation Consistency
5. Documentation Ease of Use

1.5 Compatibility Test

1. 软件自身的兼容性
   Compatibility before and after the software, the new functions of software development cannot affect the use of old functions, and cannot affect the development of subsequent functions
2. 软件对于数据的兼容性（用户数据）
   When designing functions, consider the user's existing data
3. 软件对应用平台的兼容性(安装软件的环境,硬件环境,APP,浏览器)
   APP / IOS / Android Different brands of mobile phones, different models, different web browsers, different browsers on different computer devices
4. 软件对于第三方软件或者第三方软件数据的兼容性(相关软件)
   Taobao/ Alipay/ Weibo/ third-party login

1.6 Usability testing

user experience testing

1. 标准性和规范性
   All kinds of information on the interface should conform to the norms and habits, otherwise the user will be uncomfortable to use and will not be recognized by the user. Testers need to report problems inconsistent with standard specifications and habits as defects
2. 直观性
   User-desired actions are within the user-visible range
3. 灵活性
   Keyboard: Jiugongge, full keyboard, handwriting, pinyin
4. 舒适性
5. 实用性

1.7 Installation and uninstallation test

1. Different installation paths, install and uninstall software normally
   APPstore, installation package, third party (software assistant), command line, etc.
2. Whether to pause during the installation or uninstallation process, after the pause, it can still be installed and uninstalled normally
3. Insufficient space during installation, there are prompts
4. Uninstall the software normally, if the uninstallation is canceled, the software can be used normally (data recovery)
5. If there is an exception during the installation process, the software can handle it normally (power failure, abnormal network connection, etc.)

1.8 Security Test

Security refers to information security, software protection of user data, implicit, and the security of the data transmission process, preventing virus intrusion and hacker attacks.

1. Input fields, such as entering malicious or virus-laden scripts or long strings
2. Security issues in the code, such as SQL/XML injection
3. Insecure data storage or transmission
4. Data files, mail files, system configuration files, etc. contain information or data that endangers the system;
5. Problematic access control, permission assignment, etc.
6. Fake ID: Identity Spoofing
7. Tampering, malicious modification of data, breach of data integrity

For security testing, there are ways of code reading and tool detection

1.9 Performance Test

系统性能问题的表现：Resource leaks; unbalanced resource allocation; thread deadlock; query speed is getting slower and slower; response is getting slower and slower
性能指标：TPS (transaction processing per second), HTTP requests per second, click rate, throughput, response time, CPU and resource utilization

1.10 Memory Leaks

内存泄漏产生的原因：There are problems written in the program, and there is no way to reclaim the memory; the API function is used incorrectly, and the memory cannot be reclaimed; after the memory is allocated, the memory is forgotten to be reclaimed.

检查代码是否资源泄漏：manual inspection, tool inspection

2. Divide according to whether to view the code

2.1 Black box testing

Don't care about the logical structure inside the code, don't look at the code, only care about whether the external input and output of the software function meet the needs of users (equivalent to shielding the internal implementation of the software)

advantage:

1. No need to care about the internal implementation of the software, no need to look at the code
2. Designing test cases from the user's point of view is easy to cultivate product thinking
3. Design test cases are designed according to software requirements, and it is not easy to miss requirements

Methods of black-box testing:
equivalence class, boundary value, scenario method, causal diagram, error guessing method, orthogonal method

2.2 White box testing

Check whether the code is standardized, whether the code style is consistent with the company's design, analyze the logical structure of the code, and test the code to see if the code meets the requirements.

White-box testing methods:
statement coverage, path coverage, logic coverage, decision coverage, condition coverage, decision combination coverage, decision and condition coverage, condition and condition combination

2.3 Gray box testing

Not only care about the input and output of software functions, but also care about the realization of software internal programs

3. According to the development stage

test pyramid

Features:

1. The more you go to the bottom, the higher the test efficiency
2. The further you go to the bottom, the easier it is to locate the problem
3. The more you go to the bottom, the higher the test independence and the lower the coupling

3.1 Unit testing phase

It is worth testing the smallest unit of software to see if the function of the test unit is normal

1. Testing phase: After general unit testing
2. Test object: interface between modules
3. Tester: white box test engineer or development engineer
4. Test basis: unit test module + outline design document
5. Testing method: Black box testing combined with white box testing
6. Test content: interface test, local data structure test, path test, boundary test, loop test, error handling test, local variable test

3.2 Integration testing

Combine the unit modules according to certain logic and strategies to form a large module with complete functions.

1. Testing phase: After general unit testing
2. Test object: interface between modules
3. Tester: white box test engineer or development engineer
4. Test basis: unit test module + outline design document
5. Testing method: Black box testing combined with white box testing
6. Test content: data transmission between modules, function conflicts between modules, correctness of module assembly functions, global data structure, and the impact of single module defects on the system

3.3 System Test

Conduct comprehensive functional and non-functional testing of software systems

1. Testing phase: after integration testing
2. Test object: the entire software system
3. Tester: black box test engineer
4. Test basis: requirements design document
5. Test method: black box test
6. Test content: system function, interface, reliability, fault tolerance, usability, portability, compatibility, security, performance, installation and uninstallation (new software)

3.4 Regression testing

When the system introduces new code, testers often need to verify the impact of the new code on the old functions, and the tests done are called regression tests. (When adding new features -> modify BUG)

3.5 Smoke test

After the software development is completed, the basic functions and core processes of the software must be tested. After the test is passed, the formal test environment can be entered. If the test fails, the tester has the right to call back and let the development re-modify until the smoke is successful. .

Both regression testing and smoke testing are system testing.

3.6 Acceptance testing

The last test before the software goes live, also known as delivery test.

1. Testing Phase: After System Testing
2. Test object: overall software system
3. Tester: user
4. Test basis: user needs
5. Test method: black box test
6. Test content: same system test (documentation test), usability analysis document, requirements design document, software design document, software development document, function manual, user manual

4. According to the implementation organization

4.1 Alpha testing

conducted prior to beta testing

Invite users or non-testing and development people to the development site for testing;
test environment: development site
testers: non-development and testing people

4.2 Beta testing

Let actual users test in the actual use environment, and give unified summary feedback on the problems after the test is completed

α 测试和 β 测试的区别：
The test environment is different; the concentration of test time is different; alpha testing takes precedence over beta testing

4.3 Third-party testing

It is a software third-party evaluation organization, which tests the software according to the standards and specifications of the software industry

5. According to whether the code is running or not

5.1 Static testing

Do not run the code, check whether the style and format of the code conform to the company's specification standards, and check whether the logical structure of the code meets the required functions.

5.2 Dynamic testing

Run the code, give the program the input it responds to, and see if you get the desired output

Most software testing work is dynamic testing.

6. According to whether manual division

6.1 Manual testing

According to the test case, manually test the function of the system

缺点：Large volume is prone to errors; low efficiency; some extreme cases cannot be tested for
优点：exploratory testing, which is more flexible

6.2 Automated testing

The machine operates according to the pre-set conditions that are considered to be set. These pre-sets include normal and abnormal ones, to check whether the software system meets the set conditions.

Automated testing converts manual test cases into scripts to run.

7. By region

7.1 Software Internationalization Test

Software internationalization: When designing and developing software, use an engineering technology so that when the software is converted into a different national language, it is not necessary to modify the source code, adapt to different languages, customs and habits of people in different countries, etc. (iPhone , Huawei mobile phone, office, Word)

1. From the appearance, the interface functions are not missing and can be used normally
2. Whether it adapts to the usage habits, characters, dates, and customs of people in this country
3. unit of measure, currency, unit of weight
4. Under different resolutions, the software can be displayed and used normally
5. different hardware

7.2 Software localization testing

Specific to a certain country, the above is all about localization testing