It is only in very rare cases that software can be verified purely against the specification. Because of this, testing is very important in software development.
So far, we have not clearly stated the role of testing in the software development process. This is not because we believe testing is unimportant or secondary, but because it plays essentially the same role in MDSD as it does elsewhere.
Test automation is a key element in making it continuous and repeatable during software creation. It is not necessary to act as a pioneer in testing, but rather to focus on the details in the general MDSD environment and in the specific architecture center MDSD.
1. Test type
Discussions of what aspects of a software system are tested, when they are tested, and what the end result is lead to the testing types.
- Component tests (also known as unit tests) are created by developers in the course of their day-to-day work to ensure the correct functioning of parts of the system.
- System testing ensures the correct interaction of the various parts of the system.
- Acceptance testing shows whether the system makes sense from the customer's point of view and meets their needs.
- GUI testing can take the role of acceptance testing or integration testing.
- Non-functional tests verify the requirements of the target architecture, such as security, correct transaction management in a multi-user environment, or robustness against hardware failures.
- Load testing is a special subset of non-functional testing.
- Regression testing is used to detect unwanted side effects due to changes to the source code. Regression testing is usually performed by repeating other test types.
Test automation is a very important way to increase testing efficiency and effectiveness. Only reproducible, automated testing forms the "safety net" that allows software to remain mutable. Test automation helps with regression testing and applies to all testing except ergonomics.
2. Testing in model-driven application development
In the application development thread, the same test types are relevant as in the non-model-driven process. A model is a piece of code. Because they are official and automatically converted to 3G L codes through conversion.
A model-driven approach has many potentials and can also simplify the creation of test code. Black-box testing always compares the current state with the desired state: no distinction is made between method return values and side effects. Thus, the test protects the semantics hidden behind the cleansing defined by the syntax.
1. Unit testing
In a conventional software development environment, small but useful frameworks such as Junit are available. We hope to continue to use these tools in MDSD and to support the efficient creation of tests using model-driven technology skills.
1) Separation of test cases and test data
2) Test infrastructure generation
3) Constraint check
4) Generation of extreme value tests
5) Imitation object
2. Take the test
These tests must be defined independently of the application model. Generating acceptance tests against the same model from which the application was built does nothing because it will never fail in this case as long as the domain architecture is correct.
3. Load test
In principle, load testing follows the pattern of simulating a set of clients typically running on a large number of computers. These clients run test scripts.
Environment settings (eg servers, networks, databases) must be created manually. The following aspects must be defined:
- test script
- Number of clients
- Their internal parallelism, that is, the number of calls and threads.
-
In most cases, the latter two aspects are handled by tools that also measure timing behavior and monitor the application. Test scripts are well suited for generation. Sequence diagrams or state diagrams are good foundations.
4. Non-functional testing
Testing non-functional requirements, such as reliability, transactional consistency, or degrees of freedom, cannot be done in a simple environment, such as in Junit. A realistic situation is needed here. Network or database failures must be simulated, or forced to actually occur. Similarly, security testing can only be directed manually. Simulation-driven development does not provide any specific help here.
5. Model confirmation
Model validation is a type of testing that is only available when using MDSD. It opens up entirely new options for testing. 3 different subtypes must be distinguished:
- Acceptance tests at the model level to verify the semantics of the model
- Well-formed tests for checking compliance with modeling rules
- simulated simulation
1) Acceptance testing at the model level
MDSD models are validated by communicating directly with customers or specialists about potential content, especially when the MDSD domain and its DSL are business oriented. This approach can provide additional certainty, especially when creating code or configuration files from models. Therefore, the meaning of such code or configuration can be checked in advance at a more abstract level.
2) Well-Formed Test
From a testing point of view, such modeling rules are invariants that are valid for all instances of the DSL -- that is, for all models. They must be checked before the actual MDSD conversion can be done, otherwise the conversion result is ambiguous. Well-formed tests can be static semantic checks of programming language-like languages by modeling tools or subsequent compilers.
3) Simulation of simulation
If the dynamic properties of the system are fully described in the model, they can be verified by simulation of the model. This approach is very popular in embedded systems, but only for specific types of behavior definitions, especially finite state automata. UML2.0 and Action Semantics also help with this. In common practice, simulation methods are not economical or feasible because the dynamic system behavior is not specified in the model.
3. Test Domain Architecture
Domain architectures are also software and therefore must be adequately tested. Fortunately, this kind of problem can be broken down into individual aspects for which well-known test solutions are available.
1. Test reference implementation and MDSD platform
The reference implementation plays a central role -- at least in the bootstrapping phase of the domain architecture. A flexible, test-driven approach is very useful here, since there are typically small teams of experts for this purpose. MDSD platform also has the same situation,
2. DSL Acceptance Test
The verification of the modeling language (DSL) is also very important. Validation occurs during the use of the modeling language in the referenced model. The latter is primarily a test of the DSL in terms of its usability and ergonomics. Because the referenced implementation and referenced model cover all constructs of the DSL as minimally as possible -- a minimal test that fully tests each feature once -- the referenced model should be considered a fairly important test. After bootstrapping, the application model from the development thread is the next DSL test - the real application model can be used as a test case for the suitability of the DSL.
3. MDSD conversion test
In the architectural development thread, implementation refers to formalizing aspects of the implementation and coercing it into a computer-processable form—mainly generator templates or similar transformation rules that bind the DSL metamodel. Usually, transformation rules are built on the general MDSD tool, which is assumed to be correct for our goals. Now only the domain-specific and platform-specific transformations need to be tested.
The rather obvious testing approach is a by-product of the bootstrap domain architecture: because transformation rules are derived from both the reference implementation and the reference model, they can replicate exactly some part of the reference implementation included by the DSL. In other words, if you apply a newly created transformation to a referenced model, you can obtain - depending on the scope of the architecture - either a complete reference implementation or just its implementer skeleton.
During the course of the project, the initial tests of the domain architecture are extended to test the actual application, so the created application will implicitly validate the architecture -- that is, by the application's tests. This is particularly effective and sufficient in practice.
Displayed conversion test
This approach, and the construction of the respective test suites, is perfectly feasible when constructing generators from modules.
The test provides specific stimuli to the system under test and compares the output or domain specifications of the executing system. Abstract in terms of the implementation of the system under test.