ICLR 2020, the top deep learning conference, will be held in Addis Ababa, the capital of Ethiopia, on April 26 next year.
Recently, ICLR 2020 was released and a total of 687 of the 2594 papers were accepted, including 48 orals, 108 spotlights, and 531 posters. The acceptance rate is 26.5%, which is a slight drop from last year's 31.4%.
According to statistics, there are about 320 accepted papers involving Chinese scholars, accounting for 47%.
→ ICLR 2020 official review page
It can be seen from the official blog post that this year’s review process remains basically the same as last year, but two adjustments have been made: the conference organizing committee does not allow public comments during the review period, so that the authors and reviewers can discuss the main points of the discussion. Have a clear understanding, and leave a week for replacement and emergency review.
In addition, in order to help reviewers make clearer decisions, this year’s scoring system has also been simplified. The organizing committee has cancelled the option of neutral scoring: only “reject”, weak reject, and weak reject are retained. Accept (weak acceptance), accept (accept)" four options.
The average score is generated by assigning scores to the four scoring options mentioned above. For various reasons, the scores are distributed asymmetrically, with 1, 3, 6, and 8. For accepted papers, the organizing committee is worried that reviewers are unwilling to give 10 points. Actually, for authors, they may want the paper to be accepted, but it does not mean that it should get 10 points. In addition, the large gap between 3 and 6 points allows a greater distinction between weak rejection and weak acceptance, in order to provide more suggestions to avoid "neutrality." Although these practices are somewhat atypical, the average value obtained is still meaningful for the final guidance decision.
For ICLR, with the significant increase in the number of papers submitted each year, the task of establishing a large pool of experienced reviewers has become increasingly difficult. This year, a total of 119 ACs and 2,200 reviewers participated in the review work. Although the number of reviewers was not as large as the committee's initial expectations, the burden on the reviewers exceeded expectations, but the quality of the final accepted papers was very high. .
In addition, ICLR2020 also rejected a small number of papers (less than 20) because they violated the "dual submission policy." At the same time, a large number of papers were withdrawn, such as papers that were approaching and overlapping with other conferences' deadlines, or were inconsistent with the "anonymity policy" of other conferences such as ACL.
1. Explain the OpenReview data in detail
After the ICLR 2020 paper selection results were announced, the younger brother Sun Shao-Hua, a doctoral student majoring in computer science at the University of Southern California, conducted a detailed analysis of the submitted paper data. In addition, one of his papers was accepted as a spotlight paper.
→ The OpenReview data upload page
1. Paper Scoring
From the histogram of ICLR2020 paper scores, it can be seen that the distribution of reviewers’ scores is concentrated around 4 points (average is 4.1822 points). Among them, there are 367 papers with 4 points; 266 papers with 5 points; 34 papers with 8 points.
③62% of the papers with a score of 3-6-8 were accepted;
④ Only 27.8% of papers with a score of 3-6-6 were accepted.
The cumulative sum of reviewer ratings is shown in the figure below:
2.ICLR 2020 keywords
The following figure is a word cloud image generated from the keywords of the submission, highlighting hot topics such as deep learning, reinforcement learning, representation learning, generative models, and graph neural networks.
The frequency of TOP50 keyword terms is shown in the figure below:
According to the average score of reviewers and the frequency of keywords, if you want to increase your chances of getting a high score, you can use more keywords, such as in-depth learning or gradient descent.
3. Length of review
The average review length of ICLR 2020 is 407.91 words. The histogram is as follows:
4. The reviewer's rating changed during the rebuttal period
The changes in everyone’s ratings are shown below:
The average score change of each paper is shown in the figure below:
5. Full score essay
There are 34 papers with full scores on ICLR2020, many of which come from domestic universities such as Tsinghua University, Shanghai Jiaotong University, Harbin Institute of Technology, Xidian University, and domestic companies such as Huawei and ByteDance.
Papers with an average score of 8 points:
Second, the review cited disputes
ICLR, the full name of International Conference on Learning Representations (International Conference on Learning Representations), was founded in 2013 by Yoshua Bengio and Yann LeCun in the three mountains of deep learning. As we all know, Yoshua Bengio is in charge of the University of Montreal Artificial Intelligence Laboratory, also known as MILA, which is one of the largest artificial intelligence research centers in the world. Yann LeCun is the dean of the Facebook Artificial Intelligence Research Institute and is known as the father of convolutional neural networks.
Although ICLR has only been established for six years, it has been widely recognized by academic researchers and is regarded as the "top conference for deep learning." ICLR adopts the Open Review review system. According to regulations, all submitted papers will disclose their names and other information, and accept all peer reviews and questions (open peer review). Any scholar can evaluate the paper anonymously or in real name. After the public review is over, the author of the paper can also adjust and modify the paper.
The review of ICLR 2020 has received a lot of complaints and controversy earlier. For example, Professor Zhou Zhihua of NTU revealed that 47% of ICLR 2020 reviewers have never published a paper in this field; for another example, after an ICLR 2020 paper received a perfect full score evaluation, the other two reviewers The manuscript gave two consecutive 1-point evaluations, and the three reviewers of the papers all gave high scores of 6-6-6, but AC made comments that did not apply to their own papers, which all triggered intense discussions.
After the list is released, there are always some people happy and some worry. In short, congratulations to the selected students! Has your paper been selected?
Author | Academic King
Typesetting | Academic Youth
Proofreading | Xiao Man Zixuan
Responsible Editor | Academic Youth Excellent Academic
Past review:
Which computer major is better than the four world university rankings such as THE and QS?