Preface
In June 2020, by coincidence, I participated in the CMMI3 review of my company as an ATM. The evaluation lasted about 8 days from the beginning to the end. Although the process often required overtime because of the daily work on hand, it was quite rewarding while feeling a little tired. So I decided to record this process as a summary for myself.
Some general words
Because it was during the epidemic, the online assessment method was adopted this time. Before the start of each day, the assessor will post the link and meeting number of the Tencent meeting in the group, and ATM, translators and related interviewers will enter the online meeting room to carry out the assessment together with the assessor. I have to say that this online approach has greatly eased my nervousness as a novice in this area.
Let me talk about the evaluation environment first. Although it is an online evaluation, the four ATM members are still in a separate meeting room of the company and use their computers to enter the online meeting room. At the same time, the interviewee needs to be in the same meeting room as the ATM.
Then talk about the principle that the appraiser has emphasized many times, the principle of non-attribution and the principle of confidentiality. The former means that any problems are found in the evaluation process, and the specific project names and names cannot be reflected in the description of the weaknesses, so as to avoid the company from imposing liability on the employees. The principle of confidentiality refers to the assessment documents, the progress of the process, and the opinions of the ATM and the appraiser in the process are limited to the exchanges between the appraiser, the interpreter and the ATM members.
Process record
On the first day, mutual understanding, simple training, grouping of members, preliminary inspection of documents.
Knowing each other: The appraiser first asked the ATM to introduce himself. Such as what is your name, how long you have been in the company, what your position is, whether you have a superior relationship with the interviewee, whether you can be sure that you are objective, and why do you think you choose you to be an ATM, etc. For each person's self-introduction, the assessor will find out if there is any problem. At the same time, the assessor will briefly introduce his own situation.
Simple training: The evaluator trained the ATM on some key rules of the evaluation, introduced the following basic process and work arrangement, and sent the training courseware and the PDF of the CMMI3 model to the ATM for learning.
Grouping of members: The assessor divided the ATM into two groups based on their respective assessment experience. During the evaluation process, each group is responsible for a different PA, and the group is further divided, and each person is responsible for a different PA.
Document preliminary inspection: The two ATM teams conduct preliminary inspections of the documents submitted for evaluation in their respective PAs. The main purpose is to check whether the link to the document is valid and correct.
On the second day, ATM training, document review, and initiator talk.
ATM training: It is still training ATM members. The working methods and habits of each appraiser, as well as the requirements for appraisal are likely to be different, so appraisers will pass training to convey some of his requirements for ATM in document inspection and PIID form filling.
Document check: Check the content of the document on the basis of checking the document link the day before. It is still a group as a unit to inspect the PA that is responsible for it, but this inspection is much more detailed than the previous one. It is necessary to record the weaknesses found. If no weaknesses are found, explain where and which aspects of the document reflect the corresponding requirements in the CMMI model. The inspection result is recorded in the designated column of the PIID table. Looking at the document in this way seems slow, so this process is accompanied by interviews and other matters, and continues until the preliminary results are released.
Initiator's conversation: The evaluator and the initiator have a conversation. Only the evaluator, the initiator, and the translator are present, so the specific content is not known. The guess is that the evaluator confirms the goal of the evaluation to the initiator, informs him of his current preparations, and asks whether the evaluation can be started, and so on.
On the third day, document review, problem preparation, kick-off meeting, and personnel interviews.
Document check: It is consistent with the work content of the second day. From then on until the initial results were released, ATM basically carried out this work when there was no interview and training.
Question preparation: Because there was the first interview that day, the assessor sent the questions he prepared to the ATM for review in advance. On this basis, ATM can add additional questions. In short, prepare to ask the interviewee if there are any questions. During the interview, the evaluator usually does not ask all the questions he has prepared, and the ATM can also choose from the remaining questions. Of course, ATM does not need to add questions for each interview, but it is recommended to prepare 1-2 questions for each interview, just in case.
Kick-off meeting: All personnel including appraisers, translators, ATMs, sponsors, coordinators and interviewed persons will participate in the kick-off meeting. Except for the appraiser and translator, everyone else attended the meeting as required. At the meeting, the assessor introduced the objectives, principles and scope of the assessment, and at the same time briefly introduced the CMMI3 model and announced the detailed schedule of the assessment.
Personnel interview: The first interview will be started after the kick-off meeting. The evaluator will ask questions, the interviewee will answer, and the translator will relay it in the middle. In this process, ATM needs to record the questions and answers in a very detailed and truthful manner.
On the fourth day, document inspection and personnel interviews. All processes remain the same.
On the fifth day, document inspection and personnel interviews. There was an interview on this day. The evaluator asked an ATM as the questioner to conduct the interview. Other ATMs did the interview record. He and the translator did not participate. After the interview is over, the ATMs that record the interview will each send their own records to the appraiser. So far the interview is all over.
On the sixth day, document inspection, summary and voting on weaknesses.
The document check is still carried out in accordance with the previous requirements. Then the evaluator announced the weaknesses he discovered to the ATM, and at the same time asked the ATM to add the weaknesses he discovered to the back of the weaknesses he discovered for summary. After the summary, the appraiser and the ATM will read the weaknesses found one by one and discuss the weaknesses to ensure that the final presented weaknesses are recognized by the appraiser and all ATMs. If you object to the weakness, you need to provide evidence to overturn it. But in fact, the first is that the time for this process is limited, and the second is that the appraiser has much more experience than ATM, and most of the weaknesses he found are irrefutable. In addition, the description of some weaknesses will also be discussed and revised.
On the seventh day, the preliminary findings were released, the strengths were proposed, and the goals were reached and discussed.
The preliminary findings are basically a list of the weaknesses found in the evaluation process. The evaluator will go through the PPT and ATM before release to check whether there are any content or expression problems to ensure that the content released is accurate. The preliminary results were released twice, the first time for the EPG team leader, and the second time for all participants in the interview. In both releases, the assessor asked two ATM members to read aloud the weaknesses. After the results are released, the assessor will ask the interviewed whether there are any objections. If there is evidence to prove that a certain weakness does not exist, it must be provided to the assessor and ATM members within the specified time. The weaknesses we discovered this time are not serious. The assessor specifically pointed out that if there is evidence at hand, it can be provided. If it takes a lot of time to search, it is unnecessary.
Proposal of strengths: According to the requirements of the appraiser, ATM confirms whether it has found any strengths during the entire interview and inspection of documents. If they do exist, they can propose them. The appraiser and all ATM members need to vote on the strengths proposed.
Goal achievement discussion: In this link, the evaluator still trains ATM members first, telling under what circumstances to directly determine the goal is reached (all FI or only one weakness), and under what circumstances need to discuss whether the goal is reached (the same SP) Or there are 2 or more weaknesses in the GP), under what circumstances it is directly judged as not meeting the target (pi or NI appears). At the same time, it also introduced in the second situation, from which aspects ATM should consider whether the goal is reached (mainly look at the set of weaknesses in the same SP or GP to see whether it will affect the achievement of the goal).
In our evaluation this time, the last situation did not happen. After the appraiser introduced the rules to the ATM, the ATM, based on the preliminary findings, color-coded each SP and GP in the excel form sent by the appraiser. Through different colors, it can clearly reflect which are the determined goals to be achieved, and which are still to be discussed. Then the appraiser and the ATM will discuss and vote on the SP and GP that need to be discussed.
On the eighth day, the final results were released, and the evaluation ended.
Final results release: On the basis of the preliminary findings the day before, the strengths were supplemented, and at the same time, the conclusion and summary of this evaluation were made. Also before the release, the evaluator and the ATM will go through the PPT to confirm that it is correct, and then release. In principle, participants require ATM, promoters, coordinators and all interviewees to be present, but in fact, except for ATM and promoters, others can not participate if there is something to do (all except the promoters when the preliminary results are released People must participate because they need to confirm that there is no objection to the result).
The end of the evaluation: mainly some ATMs and sponsors sign, fill in the survey questions on the SAS website, and fill in the feedback on the online evaluation method. Then the evaluation of CMMI3 ended smoothly~
Finally, let me talk about some things that I feel need attention or interesting.
The evaluation is actually not rigorous in general. As long as you prepare well, you don't feel that the problem is big. During the interview, if you mention materials that the appraiser did not find, or are asked about documents that the appraiser did not find, you need to add them in time through the coordinator. In the words of ATM, keep in mind that you have to show an objective attitude, and basically follow the pace of the assessor.
Although the ultimate goal of the company owner to participate in the CMMI3 review is to enable the company to pass the evaluation smoothly, for ATMs, we must not show this obvious tendency in the evaluation process. Because the appraiser’s concept is that the purpose of appraisal is to discover the company’s weaknesses and enable the company to recognize these deficiencies, so as to improve them and achieve better development, not simply to obtain qualifications through appraisal. The positioning of ATM in the evaluation is a very objective role, and it needs to discover the existing problems to help the company make progress. If the release of water is too obvious, such as not actively looking for weaknesses, starting to refute without evidence, or being found to communicate with other non-ATM members during the evaluation process, etc., it will give the evaluator a very bad impression, and there is a change. ATM, the possibility of impact assessment.
At the end of the interview, the evaluator will definitely ask two questions that are not related to the interview: What do you think the company needs to improve? Give an example; where do you think the company has made significant progress in the past year? for example. Before the release of the final results of these two questions, the assessor also asked ATM members to fill in these two questions. In the end, everyone’s suggestions and items that have been significantly improved in the past year were anonymously included by the evaluator in the PPT of the final release to inform the initiator, which can be regarded as helping employees to give feedback hahaha.
Conclusion
The above is just a personal experience. The evaluation process will vary according to different company situations and different appraisers. Before the evaluation started and when the evaluation just started, I was still very stressed, and I was afraid that dropping the chain would have a bad impact on the evaluation results. But in the end, just like other problems, I was full of fear when it didn't come, and when it passed, I felt okay. To be honest, I am very fortunate to have such an experience, which allowed me to really come into contact with new things and have a better understanding of these cultural concepts. Come on in the future~