Soft labor practice - the third job Sudoku test summary
1, a flowchart of the test
2, the test state analysis
State of the program to enumerate https://github.com/numb-men/software_test/blob/master/src/main/java/cn/hengyumo/SoftwareTestStatus.java
Program log https://github.com/numb-men/software_test/blob/master/log.json
1, the preparatory action may arise status log
// 没在共享文档填写仓库地址，按照老师预先声明的软工实践记分规则视为作业没交/-18.0
EMPTY_GITHUB_REPO("空，未填github仓库地址"),
// 仓库地址不符合规范 http://github.com/xxxx/xxxx
// （这一错误我能改的都帮着改了）
BAD_GITHUB_REPO_URL("不合法的github仓库形式"),
// 进行下载
WAIT_TO_DOWNLOAD("链接无误，等待下载"),2, downloading codecs action may occur status log
// 下载成功，部分同学没有按照作业要求忽略仓库的无用文件，
// 导致仓库好几十MB，有两位同学一个四十几MB，一个六十几MB...
// 实际上只上传源代码，应该就只有几KB。
// 进入解压
DOWNLOAD_SUCCEED("下载成功"),
// 下载失败，这一错误代表着，你在共享文档填的仓库地址不存在
// 或者有的同学建了github账号，却没有上传仓库，而填仓库，那访问肯定404了
// 或者有的同学填了仓库地址，但实际上"http://github.com/后面那个用户名"都无法访问
// 这种情况代表着该同学未建立账号
DOWNLOAD_FAIL("下载失败"),
// 保存的时候失败，未出现
SAVE_FAIL("保存失败"),
// 解压的时候失败，未出现
UNZIP_FAIL("解压失败"),
// 解压成功，进入编译
UNZIP_SUCCEED("解压成功"),
// 未知错误，未出现
UNKNOWN_ERROR("未知错误"),
// 连接超时，未出现
TIMEOUT_FOR_CONNECT("连接超时"),3, log compiled state action may occur (does not mean that the state has executed)
// 这一错误是未找到主程序 Sudoku/Suduku/sudoku/suduku (后缀是.c或者.cpp或者.java)
// 这些是测试程序会尝试查找的主程序文件名，作业要求中已经明确规定了项目结构。
// 这一阶段失败，编译中止
MAIN_PROGRAM_FILE_NOT_FOUND("主程序未找到"),
// 查找到主程序，等待编译
WAIT_TO_COMPILE("等待编译"),
// 编译查找文件出现IOException，未发生
COMPILE_DIR_OR_FILE_NOT_FOUND("待编译文件夹或文件未找到"),
// 编译C/C++失败，详情可以查看编译日志
// 失败的原因，大致有这些：
// 1、头文件引入失败：程序所需的.h/.cpp文件未包含在主程序所在文件夹
// 2、使用了不安全的函数，如fscanf_s等导致编译失败
// 3、程序本身语法错误，导致编译失败
COMPILE_CCPP_FAIL("编译C/C++失败"),
// 转入测试
COMPILE_CCPP_SUCCEED("编译C/C++成功"),
// java都编译过了（跨平台语言就是niu）
COMPILE_JAVA_FAIL("编译Java失败"),
// 转入测试
COMPILE_JAVA_SUCCEED("编译Java成功"),
// 该状态未使用
PACK_JAR_FAIL("打包jar失败"),
// 该状态未使用
PACK_JAR_SUCCEED("打包jar成功"),
// 使用了除c/c++/java之外的编程语言写了代码，未出现
UN_SUPPORT_COMPILE_TYPE("不支持编译类型"),4, state test action may occur
// 测试所有用例的总时间 > 5分钟，视为超时
// 有查看部分超时同学的代码，是因为用了cin或者system paues导致的缓冲区阻塞
// 还有部分可能是出现了死循环
TEST_TIMEOUT("测试超时"),
// 编译成功的项目，进入准备测试
WAIT_TO_TEST("准备测试"),
// 测试失败（所有测试用例全部失败）
// 原因可能有：
// 1、代码本身出错，没有处理好输入，或者其他原因，编译的exe或class运行失败
// 2、使用过高的vs版本，或者过低的vs版本，导致的不兼容
// 3、没有按照要求根据-i、-o读取输入输出路径，导致程序没有正确的输出结果
TEST_FAIL("测试失败"),
// 恭喜，测试成功
TEST_SUCCEED("测试成功");5, attached manually test phase
The main purpose of this stage are:
- The test phase timeout scoring program
For the incompatible version of the program appeared to try manual compilation vs 2017, re-test.
I and a classmate after failing to build a chat, I find windows sdk version of vs and his is not the same, this is because the most recently updated vs, vs some of them are experiencing this problem, or just under recently updated. Is that wrong than one expected. I had to update and re-vs compile and test it again. Tao here to thank the students to help me find out why. According to the retest is run appears:
However, lead windows sdk version does not just cause of the error vs shown in the picture appears. More is not the version vs 2017
(using 2017 which is working to emphasize, it is impossible to who specifically again next 2019,2013,2015)
This phase of compilation state and, consistent test phase, is omitted.
6, summary stage
The test output of the table, the results of binding attachment test phase, to generate the final result.
How do you see the error of their own program
Program compile / test output log: https://github.com/numb-men/software_test/tree/master/log
which
compile.log.main.txt - a compilation log, you can view your program based on your search for student number the command-line compiler output
test.log.main.txt - test log, you can view your program to test the command-line output based on your search for student number
Test Case
https://github.com/numb-men/software_test/tree/master/test_case
https://github.com/numb-men/software_test/tree/master/test_case2
test_case
├─3
│ input1.txt
│ input2.txt
│ input3.txt
│ input4.txt
│ input5.txt
│ output1.txt
│ output2.txt
│ output3.txt
│ output4.txt
│ output5.txt
│
├─4
...
// tase_case 中包含7个文件夹，分别存放3-9宫格的输入和答案
// 每个子文件夹，存放五个输入和五个对应的答案
// 其中input1 input2 input3都是单个表盘，input4 5个表盘，input5 10个表盘Test a total of 5 * 7 = 35, due Sudoku delimiter before Xu assistant described in the release operation is a space,
but provides us with the test cases are the two spaces, program assistant job in order to prevent errors caused by error,
so every program will use a space test again, two spaces test again and the results were correct that.
Scoring criteria
3 grids, each test case 5 points out of 5 * 5 = 25
4-9 grids, each test 0.6 points, out of 0.6 * 5 * 6 = 18 points
3-9 grids are all correct points :2 minutes
Third job: job score 55 (blog) + 45 (program) = 100, converted thousands sails compete FIG scoring 40 points;
therefore this test program job score out of 45 * 0.4 = 18 minutes
Many students are very small error (does not meet the directory structure / use of system pause, cin as a result of obstruction), got zero points for pity, why not re-test again, give another chance?
Ran cited the words of assistant:
My advice is now rating the results remain the same. There are questions about the performance of students, they can in the time (such as grades posted 24h) initiate such representations, assistant group Li Aite request to re-automated test procedures. Is not submitted in accordance with requirements of the job, they do not give points according to the requirements. I personally as a student and as an assistant on the sidelines of the lessons of experience is only enough pain to make the students really remember this lesson and improved. Another point of view, to allow students to resubmit work, again in accordance with the rules of the job to complete the job, the job results can perhaps be improved, but the students next job, the next job at it? In strict accordance with the requirements of the job will be done right? Given the opportunity to resubmit the job, I would like for students more harm than good, since I'm not in accordance with the requirements, anyway, many students will be so, then the teacher will give the opportunity to resubmit the job. So again, a vicious cycle. Zero is not bad, there are negative points. The first time such a result, it is estimated there will be students' revolt ", but after the operation, I think it would be good. For students have opinions, can be re-automated test job in accordance with their requirements, and to see whether there are differences or current results are given why the points. Repeatedly to chance, is not conducive to job requirements after, there is no advantage for students, perhaps scores have improved, or teaching assistants is a good man, but it lacks significance.
At last
The results of this work may not be a lot of people satisfied. But I hope you understand: the score is only a means of teaching is the goal. I hope you gain something in this job. (If you can go back and complete their work once, surely you will reap more)
The code is automatically tested using java to write, a total of about 2000 lines of code, because time is limited, there are still insufficient. If there are students interested in automated test, then welcome to join me, along with maintenance of this code is: https://github.com/numb-men/software_test
last of the last
My score is negative is not necessarily hang Branch, I'm so desperate, then I do not want to write a job:
Ran assistant: Mo panic, the final score is mapped to the same large range, then the gap will not be great unless very good. Little effect on the global single point.
(Ps: to score any objection, please 24h inner bottom blog comment reply)