EBSE special serials are divided into "five" chapters. This article is the "third" chapter of this serial series. In the previous "Chapter 2", EBSE Step 1: Case Study on Strengths and Challenges has been analyzed. In this chapter (3), we will elaborate on the "EBSE step two, determine the improvement plan through systematic research" and other content in combination with specific research practices.
https://blog.csdn.net/NewCarRen/article/details/130137367?spm=1001.2014.3001.55025. EBSE Step 2: Determine the improvement plan through systematic research
This section describes the purpose and design of our approach to improving the testing process in a systematic literature review and value stream mapping. The research question for this part is RQ3: What improvements are suggested in the literature for the automotive testing process based on practical experience?
In Section 4 of the special series (2) , we identify the challenges associated with software testing in the industry. In this section, we introduce EBSE step two. We perform domain-specific systematic literature reviews to investigate the state of the art. Second, we create solution proposals based on the review results to address the identified challenges. This is done in the spirit of evidence-based software engineering: a library of evidence needs to be consulted when creating diagnostics and solution recommendations.
5.1. Systematic literature review (SLR)
The purpose of our SLR is to identify test-related problems in the field of automotive software, and solutions that have been proposed and applied in the industry. Our SLR design includes the following steps. Our SLRs were based on guidelines provided by researchers, but we did not exclude quality-based studies, as our goal was to identify all potential solutions based on industry experience and not discard them due to lack of reporting processes, etc.
Our steps in researching the literature are:
• Define the research question for the review
• Dissertation identification
• Study options
• Mapping of solutions and identified challenges
5.1.1. Paper identification
In this step, we formulate the search terms so that they identify research papers. The search terms were verified in detail in several test searches in the digital library. For this, we used five different search strings (see Table 6). The first two strings identify articles about testing in the automotive domain, and model-based tools supporting automotive software development, to cover solutions in the domain of testing-related challenges. Identified requirements issues are very common requirements issues, but have implications for testing. Therefore, these are also included in separate search strings. Given that some projects are conducted in an agile way of working, which is considered a strength, we also looked for research related to automotive agility.
Table 6: Search Strings
Search strings were applied to titles and abstracts in IEEEEXplore, ACM Digital Library, Springerlink, ScienceDirect, and Wiley Interscience databases. We did not apply search strings on the full text, as we found that this approach often yielded too many irrelevant results.
5.1.2. Study selection
In order to select papers relevant to our objectives, we developed inclusion/exclusion criteria. First, we excluded non-English papers published before 2000 (given that cars contain a lot of software in recent years, the challenge is more relevant to recent research) and articles without full text. Since our goal was to look for problems and solutions provided in the peer-reviewed literature, we excluded editorial notes, comments, etc. Since we intend to look for solutions applied in the industry, we include papers in which the solutions are empirically evaluated in industry, especially in the field of automotive software. A major criterion for included studies was that they presented solutions to problems related to software testing. Software testing refers to any V&V activity that spans the entire software development life cycle (requirements verification, test case generation, unit or regression testing, etc.). To ensure that these criteria were met, papers were filtered against the checklist.
• Is the paper in English?
• Is the full text of the paper available?
• Was the paper published in 2000 or later?
• What is the background for studying the field of automotive software?
• Does the paper discuss any problems and solutions or tools related to any software V&V?
• Does the paper include an empirical assessment in an industry context?
A search for SLR 1 and SLR 2 yields 221 papers on SLR 1 and 66 papers on SLR 2. Table 7 shows an overview of the distribution of the main studies across the database and also shows the number of studies that were finally selected.
Searches for SLR 3, SLR 3, and SLR 4 yielded 301, 12, and 107 papers, respectively. Table 8 gives an overview.
5.1.3. Solutions based on systematic research
We mapped the challenges identified in the SLR and the solutions offered to the challenges identified in the interviews. In addition, we state other references where challenges were found in this study. These contents are shown in Table 9. Based on our SLR, we came up with seven solution suggestions. It should be noted that it is often not possible to have a single solution for every problem. It all depends on the type of project and what strategy (eg resource management, budget management) the team adopts to implement these solutions.
The number of references related to a solution suggestion depends on the availability of information. When we created categories, our goal was not to have one category that needed small/simple solutions and another that covered large areas of solutions. Even though there is only one reference for Test Management (SP7), we think the focus of the research is on test management being as big a scope as test automation and tooling, looking at its complexity and how hard it is to solve it. In general, the scope of the problem is the basis for determining the granularity of the categories.
SP1: Requirements Management (RM): In general, we found that good requirements are a prerequisite for good testing in automotive software development. Requirements-related issues can be solved through better requirements management, such as unclear requirements (C03_1), demand volatility (C03_2), and demand traceability (C03_3) in the serial chapter (2) of this topic . Furthermore, we can understand that the quality attribute specification issues (C08_2) as well as customer communication issues (C06_1 and C06_3) can be improved with ideas from the requirements engineering domain. Our domain-specific SLR is able to find many solutions to these problems (see Table 9). For example, Grimm from DaimlerChrysler1 recommends simulating requirements early and deriving test cases from specifications, and recommends tracking and managing requirements throughout the software development life cycle; the researchers proposed and evaluated an approach in which text-based The use cases are elicited in interviews from different stakeholders to elicit and specify user requirements for automotive software; researchers propose abstraction levels for software, functions, systems, and vehicles. Each requirement at each level of abstraction is in turn linked to system goals and scenarios.
Table 7: Number of selected studies (SLR 1, SLR 2)
Table 8: Number of selected studies (SLR 3, SLR 4, SLR 5)
SP2: Capacity Management (CM): Identify capacity management as addressing a number of challenges. We identified competency management needs based on lack of dedicated testers (C04_1), lack of testers (C04_2), knowledge transfer (C05_1) and lack of basic testing knowledge (C05_2). The researchers found only one domain-specific source of SLR, and they proposed ways to involve experts to share knowledge and expertise with less experienced testers in the project, for example, suggesting workshops where testing teams exchange testing information with experts, Tools as a means of informal training.
SP3: Quality Assurance and Standards: Quality Assurance and Standards can also help with some issues. Researchers propose a waterfall process for automotive software engineering, in which quality assurance is performed through peer review of artifacts produced during the design, implementation, and testing phases. For this purpose, DaimlerChrysler has developed its own software quality management manual for its automotive projects, which helps to address issues C01_1/C01_2 related to the lack of a uniform test process definition and test planning. The ISO 26262 (Functional Safety for Road Vehicles) standard defines the artefacts and activities for requirements specification, architectural design, implementation and testing, system integration and verification. The standard also specifies the use of formal methods for requirements verification, notation for design and control flow analysis, test case generation and use of in-the-loop verification mechanisms, such as hardware-in-the-loop, software-in-the-loop (relevant to challenge C07_3 on test equipment) .
Table 9: Mapping of challenge areas and solution references (for C01-C010, please refer to the serial chapter (2) of this special topic)
SP4: Test Automation and SP5: Test Tool Deployment: Automation is clearly one of the most important issues in the industry, and there is a wealth of research describing the state of practice (C07_1). As shown in Table 9, many test automation solutions have been proposed and used in the automotive domain. For example, model-based black-box testing was proposed for systems with high safety and reliability requirements (C08_1); many works proposed evolutionary testing as a solution to the challenges of automated functional testing, and have been used at DaimlerChrysler under the name Successful implementation of tools for AUSTIN (C07_2); In addition, other types of testing and quality assurance tools are proposed, such as classification tree editor CTE, systematic approach for functional test case design; semi-automatic safety and reliability of software design process performance analysis, validated in a Volvo case study involving 52 active safety functions; and tools for resource usage and timing analysis (C08_1). Overall, it can be concluded that when it comes to test automation or tooling, there is no shortage of proposals specific to the automotive space.
SP6 : Agile Merger : Agile development methods have become popular in the industry in recent years, and can also help solve many problems encountered by case companies. Based on our interviews, we determined that the software development processes used in the case organization needed to be changed in response to changing requirements (C02_1, C03_1, C02), where agile processes are a natural fit because they provide Regular communication to clarify requirements. Agile also emphasizes collaboration and communication, which can be seen as addressing the identified knowledge transfer and interaction problems (C05_1, C06_1, C06_2, C06_3). Finally, some agile methods emphasize continuous and automated testing, which may help to solve many of the testing problems encountered (C01_1, C01_2, C09_1 and C09_2). Our domain-specific SLRs were unable to find many academic publications on agile, although agile can theoretically be linked to many problems. DaimlerChrysler has implemented Agile development and it has been observed that Agile provides flexibility, high speed of development and high quality. Researchers reporting experience using Extreme Programming (XP) on embedded products believe that TDD and unit test automation are key success factors.
SP7: Test Management: From the solution proposals identified above, it can be seen that most of the activities are related to the organization of tests and their artifacts. It was also evident from our interviews that most of the challenges identified in the case studies were more or less related to the management of testing activities. No identified research has to focus on test management activities related to the automotive domain. However, there are few articles in the literature describing the activities that test management must focus on in order to improve tests appropriate for this research context. This solution proposal is therefore developed to harmonize the solutions proposed above. Test management activities observed in our study can include the following activities.
• Test process management: Manage various activities in the test process, such as test planning, test analysis, test construction, and test execution. This activity also applies when introducing agile practices.
• Test artifact and asset organization: Reuse and maintain test artifacts such as test cases, test versions, test tools, test environments, test results, and test documentation. This activity can also be referred to as test configuration management, which manages changes throughout the lifecycle of a testing activity.
• Requirements management based on testing: Responsible for analyzing and determining requirements changes, so as to adjust testing progress and testing strategies reasonably, so as to improve test cases to meet new requirements.
• Competency Management: Responsible for assigning testers with the skills and knowledge required to perform specific testing activities.
• Defect Management: Responsible for early detection of defects that need to be effectively managed, supported by different people working together at each stage.
For more content, please pay attention to "Automotive EBSE Testing Process Analysis (4): Step 3, Reflection and Evidence-Based Problem Solving", follow Niuka.com, and learn more about automotive technology. Interested friends, you can add Niu Xiaoka WeChat: NewCarRen for more details