My teammate and I are having an hard time understanding if mutation testing can / should be done on JUnit tests referring to Swing classes.
More precisely, we are currently using Junit 4 and Assertj Swing to test the interface for an assignment on testing (with TDD) at the university. We are also using PITEST as mutation testing framework.
We built the Swing classes with WindowBuilder plugin for Eclipse, and as such we do not have much control on how the classes are structured. The unit tests are all green but when running pit a ton of exceptions are thrown from pit in these classes and the tests fail miserably. You can see the maven log on travis.
Since we are just starting to understand the testing world, a suggestion from someone who actually deals with these things would be much appreciated.
Thank you very much.
Mutation tests are used to test your test coverage, not your code. So when your tests fails it's OK and your test coverage is acceptable.
The problem is that the UI tests are hard to code and they are not very stable (at least they was so some years ago, especially for Linux). So the mutation tests provide no additional value for them. Any small change in UI code may provide a non-explainable stack trace, so you cannot say whether your tests are good or Swing is too complex and can be easilly get down (Swing throws tonnes of exceptions, so any little change can provoke one of them).
My suggestion is standard: separate your UI code from the business logic and provide normal tests for your business classes. In this case your mutation tests may be reasonable.