The content of the article is based on my accumulation of embedded software testing (black box) in the past three years. It is quite fast to say that three years have passed since I graduated. I have worked in two companies (this one is now The second company), the testing projects in the past few years have basically focused on embedded software. At the same time, they need to take into account the testing of APP side, WEB side functions, and web back-end interface (postman). However, the focus of the testing business is still on embedded software.
Why are the tests so complicated? In fact, all of them add up to just one product. Just because I am the only one testing it at the moment, I test the entire system by myself. Take a video conferencing system made by my current company. For example, the entire system consists of a hard terminal (Linux terminal), app terminal, web terminal (used for background account/conference management, etc.), and PC terminal. Students who have been exposed to embedded software testing should all understand these things. Now I will talk about my understanding and opinions on embedded software testing in the past few years. You are welcome to leave a message to add or discuss, please do not criticize~~
What is an embedded system?
An embedded system is a "special computer system that is completely embedded inside the controlled device and designed for specific applications."
Embedded systems are "devices used to control, monitor, or assist in the operation of machines and equipment."
Embedded systems can also be defined as "application-centered, computer technology-based, specialized computer systems with tailorable software and hardware, and strict requirements on functionality, reliability, cost, volume, and power consumption."
The embedded system consists of two parts: embedded hardware and embedded software. Hardware is the support and software is the soul.
Embedded systems are classified according to real-time properties and can be divided into non-real-time, soft real-time and hard real-time. Hard real-time means that the system has strict requirements on response time. If the response time cannot be met, it will cause a system crash or fatal error; soft real-time means that the system has strict requirements on response time, but if the system response time cannot be met, it will not Can cause fatal system errors or crashes.
What is embedded software?
Embedded software is software designed based on embedded systems. It is a type of computer software. It is also composed of programs and documents. It can be divided into three categories: operating system, support software, and application software. It is an important component of embedded systems. part.
To put it simply, it is a set of software installed on a PCB board with a Linux system. like this:
Characteristics of embedded software:
System kernel is small
Strong specificity
System simplification
High real-time operating system
base with hardware and software complementing each other
The similarities and differences between embedded software testing and commercial software testing (from the perspective of black box testing)
Same point:
The purpose and principles of embedded software testing and commercial software testing are the same. They are both to find bugs, track bugs, and verify bugs so that the product can realize the functions described in the requirements document and meet user needs.
difference:
Different testing environments are required.
When testing embedded software, the environment you need to prepare is a complete set of equipment, which may include power supplies, network cables, high-definition cables (various video transmission lines), switches, cameras, microphones, USB flash drives, routers, etc.
Testing commercial software may be an installation package or a test URL. All you need to prepare may be a computer or a mobile phone.
Testing thinking is different. (If there is a problem, check yourself first)
The testing thinking here does not mean the way of thinking required when designing test cases, but the thinking is different when locating bugs.
When testing embedded software, if a certain function cannot be implemented, it may be a function implementation problem, a memory problem, a hardware support problem, a driver problem, or a network problem. It may be a hardware physical interface problem.
Although it seems that these problems are nothing to worry about, if it is a hardware support problem and you throw a bug, after development and troubleshooting, you find that you have not connected the supported hardware according to the specifications, and you will not be the one who is embarrassed at that time; For another example, if the POE network port is not connected to a network cable that supports POE, the problem is caused by your unprofessionalism, etc. Since we are test engineers, we should have the professionalism of engineers to complete the work.
When testing commercial software, if a function cannot be implemented, in most cases we just throw a bug directly to the developer, along with bug screenshots, logs and operation steps, which usually works well.
Different skill points are required
Based on the experience of testing projects in the past few years , it is found that embedded software testing frequently applies computer network principles.
When testing embedded software, you need to connect network cables, perform network configuration, access switches, understand network communication, data transmission, etc. In other words, what you need for embedded software testing is (solid software testing theory and testing Method + basic knowledge related to computer network).
When testing commercial software, since the environment is not too complicated to set up, there is relatively little knowledge related to the computer network, and what is often needed is solid software testing theory and testing methods.
Embedded software testing’s own characteristics and testing difficulties
System resources are currently limited
Closely related to hardware systems
Diversified operating platforms
cross test pattern
System security, reliability, correctness of system operation results, etc.
The testing strategy of embedded software is very different from that of general commercial software.
Is it difficult to test embedded software? Which one is more difficult than commercial software testing?
I feel that in black box testing, embedded software testing is more difficult than commercial software testing. I only speak from the perspective of black box testing of embedded software. Why do I say that? I only think of the following points for now:
Different hardware platforms make testing more difficult. Since embedded software is closely related to the hardware platform, that is to say, if the same set of software is transplanted to another platform, it must be tested from beginning to end, which will increase the workload. It is larger than commercial software, the probability of errors will also increase, and the corresponding testing difficulty will also increase. I encountered a situation where a project needed to run on two platforms.
The test environment makes testing more difficult. I can say with certainty that the same set of equipment operates normally in environment A. As long as I change a switch and build an environment B, bugs are likely to appear. Therefore, experienced embedded software When test students encountered a bug reported by a customer, they would immediately ask what the environment was like at that time, what equipment was available, and what the operating steps were. If necessary, they would send all the equipment back to reproduce it.
The performance of the hardware platform makes testing more difficult. The chip performance of the hardware platform is definitely not as good as the CPU on the computer. Therefore, during testing, we often have some hardware performance problems that cause some bugs to occasionally appear. This is very important to us. For testers, it is very difficult to find an occasional bug. Tools are needed to track the operation steps and print logs.
External devices make testing more difficult. Basically, embedded software in life needs to be connected to other devices, such as USB flash drives, cameras, microphones, etc. During testing, if we find that a certain task cannot be achieved, we need to Consider from three aspects: ①: Is it a functional bug of the software? ②: Is it a problem of damage to the physical interface of the hardware? ③: Is there a problem with the connected peripherals and software protocol or driver problem, etc. The reasons have increased. Test difficulty.
The last reason why testing is difficult is that embedded software testing is relatively independent from software testing, and there is relatively little relevant information. Unlike commercial software testing, you can find test methods and methods with just a search. Some test articles. Embedded software testing is more of a practical experience, and the way of thinking for embedded software testing in each application is different.
end:
Testing is also a technically difficult position that requires continuous accumulation of experience. The biggest difference between veteran testing and novice testing is that veteran testing can quickly locate and find bugs, while veteran testing can have more testing methods and testing strategies to find bugs. , have better test management experience and business capabilities.
Finally, I would like to thank everyone who reads my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, if you can use it, you can take it directly:
This information should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey. I hope it can also help you!
