Software Testing Fundamentals - An Introduction to Non-Functional Testing

1. Interface test

1. Form interface test
2. Control interface test
3. Menu interface test
4. Interface test of special properties

2. Usability test

Main points of usability testing
 Business conformity
 Functional customization
 Integration of business modules
 Data sharing capability
 Constraints
 Interactivity

1. Control usability test

2. Menu usability test

3. Ease of use of shortcuts

4. Online help usability test

5. Auxiliary system usability test

• wizard test
• Whether the information prompt is described in understandable language
• Whether to prompt for important, destructive commands
• Is the information prompt uniform?

3. Compatibility test

1 The meaning of compatibility test The
compatibility test verifies the degree of dependence of the software and its environment, including the degree of dependence on hardware, the degree of dependence on the platform, and the degree of dependence on other software.
3 Prerequisites for Compatibility Testing

• Standards and specifications are a guarantee of software compatibility
  • Advanced Standards
     Rules that the product complies with
  • Low-level standards
     File formats and network communication protocols

4 Test points for compatibility testing

Fourth, the document test

1 Which documents need to be tested

• User Manual
• online help
• Readme file (readme file)
• Authorization/Registration Form/User License Agreement
• Guides & Wizards
• Wrap text and graphics
• Marketing materials
• Label
  2 Document test checklist
  
  Document test needs attention
• For software users, the content outside the program is also a part of the software;
• Documentation is often not given enough attention, lack of financial and technical support and testing;
• The person writing the documentation may not be an expert in software features and have no knowledge of the software;
• Since the printing of documents takes time, the problems between them cannot be repaired;
• Documentation testing is not just about text proofing, but also about errors in the program itself.

3. Installation test
1 Classification of installation test

• installation test
• run the test
• uninstall test
• encryption test

2 Notes on installation and testing

• Installation Manual Evaluation
• Automated testing of installations
• Testing of installation options and settings
• Interruption testing of the installation process
• Installation sequence test
• Multi-environment installation test
• Installation correctness test
• Repair installation test
• uninstall test

3 Test cases for installation testing

4 Test cases for running tests

5 Test cases for uninstall testing

6 Encryption test
6.1 Contents of encryption test

• Software encryption
   Serial number testing
   Decryption program testing
• Hardware encryption
   Test of dongle
  6.2 Test case for encryption test
  

4. Performance test

(1) Meaning of performance test

(2) Terms of performance test

1 Request
The client sends a request to the server to obtain data or resources such as files and pictures.
2 Response The
server sends data or resources such as files and pictures to the client.
3 Protocol

• Transport layer protocol
   tcp, udp
• Application layer protocol
   ftp, http, dns, dhcp, smtp, pop
  4 response time
• The time taken by the application system from the time when the user sends a request to the time when the client receives all the data.
• Web page response time breakdown
   Network transmission time: N1+N2+N3+N4.
   Application server processing data: A1+A3.
   Database processing time: A2.
  
  5 Online users Users
  who are using the software.
  6 concurrent users
• Refers to the number of all users who interact with the server at the same time.
   Online users are not necessarily concurrent users.
• Calculate the number of concurrent users
  • Estimates are generally based on past experience and industry standards.
     For example, the number of concurrent users in the telecommunications industry is often 1/10,000 of the number of online users;
     The number of concurrent users of OA software is generally 5%-20% of the number of online users.
  • Refer to other similar products.
  • Analyze historical data (the daily transaction volume for a year or half a year).
  • Try running it online.
    7 Virtual users
    Performance testing tools use virtual users to simulate the behavior of real users.
    8 Throughput and Throughput
• Throughput
  refers to the number of bytes processed by the server in a period of time, which directly reflects the carrying capacity of the server.
  

 From the correlation diagram of throughput and VU, it can be seen that when the throughput increases to a certain number of VUs, the software system has a performance bottleneck. At this time, the throughput no longer increases with the increase of VUs, but tends to balance.
 In the actual test, the throughput is unknown before the test. It must be tested by continuously adding virtual users to find the inflection point of the throughput, that is, the maximum throughput.

• Throughout
   Refers to the number of bytes returned from the server per unit time, that is, throughput/test time, or the number of client requests processed per unit time.
   It is an important indicator to measure network performance. Generally, the greater the throughput, the greater the value of the throughput rate. The greater the throughput rate, the stronger the load capacity of the system.
  9 Transactions per second (TPS, TransactionPerSecond)
• Indicates the number of transactions processed by the system per second, and is an important indicator to measure the processing capability of the system.
• If each transaction corresponds to one transaction, then TPS represents the number of transactions processed by the server per second.
  10 CTR (HitPerSecond)
• Refers to the number of HTTP requests submitted by users to the server per second.
• A single click may make multiple HTTP requests to the server.
   Usually the server has an anti-refresh mechanism to prevent the huge pressure caused by the refresh.
• The click rate only reflects the number of requests submitted by the client, and cannot represent the current pressure on the server, because the server can reject some of the client's requests when it cannot handle all the requests.
• CTR is synonymous with TPS if each click is treated as a commit transaction.
  11 Think Time
• Also known as "sleep time", waiting time.
• Refers to the time interval between each request when the user performs an operation.
• Load testing generally ignores thinking time, and stress or reliability testing sets a thinking time based on the actual situation. Usually think time is set to 3-5s.
  12 Resource utilization
• Resource utilization
   refers to the occupied degree of different hardware resources in the server system, including CPU utilization, memory utilization, disk utilization, network, etc.
   Compare the utilization rate of common resources in performance test. For example, if the utilization rate of CPU is high and other resources are low, it can be seen that the CPU is the bottleneck of the system.
   In the configuration tuning test, the tuning effect is judged by comparing the system resource utilization before and after configuration tuning.
• Performance counters (Counter)
   are some data indicators that describe the performance of a server or operating system. Mainly by adding counters to observe the usage of system resources.
   Counters play a key role in "monitoring and analysis" in performance testing, especially when analyzing system scalability and locating performance bottlenecks.
   When analyzing test results in a performance test, the analysis must be based on several different counters.

(3) Performance test classification
1 Load Testing (LoadTesting)

• By continuously pressurizing the tested system until it exceeds a predetermined index or some resources have reached a saturation state and cannot be pressurized any more.
• This method is mainly to find the maximum load capacity of the system and provide a basis for performance tuning.
  2 Stress Testing
• When the system has reached a certain degree of saturation (for example, the CPU, disk, etc. are already in a saturated state), test the ability of the system to process services, and test whether the system will crash.
• Generally, the system resources can reach a higher level by simulating the load and other methods.
• This method is generally used for system stability testing.
  3 Concurrency Testing
• By simulating concurrent user access, test multiple users accessing the same application, module, or data at the same time, and observe whether the system has performance problems such as deadlock and system processing speed significantly reduced.
  4 VolumeTesting
• A test to find the limit value of a certain index of a software system (such as the maximum number of concurrent users, the number of database records, the maximum load, workload, etc.) is a test goal.
  5 Reliability Testing (ReliabilityTesting)
• Or called stability test, robustness test.
• When the system is under certain business pressure, let the system continue to run for a period of time and observe whether the system achieves the required stability.
• Reliability testing generally must give a clear requirement, such as how many days the system can continue to operate without failure.
• is a test target.
  6 Configuration Testing (ConfigurationTesting)
• Configuration test
   By adjusting the system software/hardware environment, understand the impact of different environments on system performance, so as to find the optimal configuration of the system.
   This method is generally used for system tuning and planning.
• Benchmark
  test In a certain software, hardware and network environment, simulate a certain number of virtual users to run one or more services, use the test results as baseline data, and run the same business scenarios during system tuning or system evaluation To compare the test results to determine whether the tuning has achieved the effect or provide a decision basis for the selection of the system.

(IV) Performance testing process The
performance testing process is divided into four stages: test design, construction, execution, and analysis.
1 Design phase
Define the business process to be tested, the average processing volume of the business, the highest peak business processing volume, the combined business process, the overall user of the system, and the response time goals.
2 Build Phases

• Design setup and configure test systems and infrastructure, build test scripts and load scenarios using automated performance testing solutions.
• Specifically, it includes: writing scripts, enhancing scripts, and designing scenarios.
  3 Execution phase
  Including running load scheme and measuring system performance, monitoring system resources.
  4 Analysis, Diagnosis and Tuning Phase
  Mainly measure system performance and bring load testing to the next level, focus on finding the cause of the problem to help development engineers solve the problem quickly, and adjust system parameters in real time to improve performance.

5. Mainstream performance testing tools

PS: Suddenly ranked second in the test field, and was shocked to Σ(っ°Д °;)っ
This is just a study note, learning source: https://www.bilibili.com/video/BV1EJ41147fN,
interested small Friends can go and learn, but it's a little too detailed.