AI in Software Testing - Using AI to Write Test Cases

1. Status

• With the release of chatGPT, people's enthusiasm for artificial intelligence has been ignited, and it has also caused a panic that "human beings will be replaced by AI".
• Everyone knows that artificial intelligence is difficult to implement at this stage, but almost everyone, related and unrelated fields, are rushing into the research of this technology.
• It is a common belief that even if you have not sent "machines to replace human labor" in yourself, once the possibility arises, the changes it will bring will be very rapid.
• Everyone is worried about falling behind others in this transformation, which will lead to losing the opportunity or even being eliminated.

2. Testing technology can be combined with several directions of AI

• "Machine replaces manual work" , so first start with the manual content
• So from the software testing work, what are the work contents?
  • Test case writing and execution, bug location and tracking,
  • Writing test reports and other documents
  • Automation scripting
• The test case is an important document in the testing work
• Most of the current AI technologies are used in document writing, so using AI to write test cases is the first practical direction that comes to mind.

3. Try to use AI to write test cases (experimental process)

• Because this article mainly discusses the feasibility of a working mode, so directly use CSDN's chitGPT as the test object
  

• Throughout the course of the experiment, it was confirmed that using AI to write test cases is indeed feasible. However, some problems were also exposed in the process.

1. The ability of AI to connect with context is weak

• In the process of question and answer exchange, it is not smooth
  

2. The content of the answer is similar to the content that can be searched on the Internet

• Blank questions, get vague answers.
• If there is no clear instruction, the content obtained is very similar to that obtained through a search engine
  

3. There needs to be a clear instruction

• The experiment became interesting when I started to give the AI ​​a clear need
• AI's answer is already a test point that can be applied to actual test cases.
  

4. The output content is limited

• When I clarify the instructions for writing use cases, the results of AI output are already very complete
  
• But maybe because there are too many contents, an error will be reported in a while.
  

5. Unable to imagine content beyond the requirements description

• In the process of my two questions and answers, some content was intentionally removed, such as the description of the pop-up keyboard.
• When this description is included, the content of the AI's answer also includes this.
  
• If this description is removed, the content of the AI's answer will not contain this content.

4. Summary: Using AI to write test case solutions

• AI is like a mirror, the instructions provided are vague, and the results are also vague. The clearer the instructions, the clearer the results.
• For the above process, I have concluded a feasible solution

1. Program

1. Input requirements documents to AI and write instructions for use cases

• The instructions I used in the appeal experiment process are as follows, if you are interested, you can try it out:
  Write a test case for the following requirement.
  Requirements:
  1. The mobile phone number input box is limited to only numbers, and the input of characters other than numbers is unsuccessful. The mobile phone number is limited to 11 digits;
  2. Click Send Verification Code to initiate a verification code request to the server. And the client prompts "the verification code has been sent";
  3. In the password input box, only numbers or letters can be entered, and the number of digits is limited to 8-16;
  4. Click the registration button to submit the data;
  5. The user agreement is selected by default, and the user Click "Privacy Agreement" or "User Agreement" to jump to the corresponding agreement page;
  6. Click "Direct Login" to jump to the login pop-up box;
  7. When the user clicks the "X" button, the registration pop-up box is closed; the
  output content needs to include Use case serial number, precondition, use case title, use case steps, use case expectations.
  The output format is a table.

2. Obtain the test cases output by AI and verify them

2. Existing problems and solutions

1. Problem: AI has limited understanding of natural language and cannot relate to context.
   Solution: Requirements documents are manually refined to function points. Most functions are divided into several parts: display\trigger timing, style, function logic, and exception handling. Function points can be organized from these directions.
2. Question: How do you enter requirements documents for iterative projects and functions?
   Solution: Let AI understand the entire project through machine learning. But at the same time there is the possibility of privacy leakage. Or add a description to the requirements document.
3. Problem: The use case format produced by AI is not the final format of the use case.
   Solution: Make the instructions more detailed and more suitable for your needs. Convert the format by script, or manually.
4. Problem: The use cases produced by AI cannot think beyond the content described in the text.
   Solution: You can use machine learning, or add necessary function points when sorting out function points manually.

3. There are risks and doubts

1. Let machine learning, accompanied by serious privacy leakage. The more it knows about the company's projects, the more privacy is leaked. All AI is also a double-edged sword. If you want to make good use of AI, you must first handle the balance between them.
2. Recently, there have been news of a large number of chatGPT titles, and it is not so easy to use it conveniently and safely. Of course, there are also domestic ones. Which one is easy to use depends on the official launch of the products of major manufacturers.
3. Because the use of AI still requires some preliminary equipment and subsequent processing, whether the value of the output can be greater than the cost is also one of the issues that need to be considered.

This article is just some of my thoughts, and there may be inaccuracies or omissions. Everyone is welcome to discuss or correct my mistakes.

——————————————————————————————————————————————————————————————————————————————————————————————————
_ [Necessary Skills for Software Testing] I will upload test-related materials from time to time, you can click on the QR code below the article to get it~