Table of contents
"Good" from the perspective of subject development
"Good" from the perspective of research practice
Where do good research ideas come from?
In Wong Kar Wai's movie "The Grandmaster", there is a classic martial arts contest. Chairman Gong said to Ip Man, "Today we are not competing in martial arts, but in thinking." In fact, a good idea or idea is also the soul of an excellent research result. In the computer field, there is a popular saying "IDEA is cheap, show me the code", which also shows that for the computer science that values practice, the quality of an idea depends on its actual performance. Here's where to talk about where good research ideas come from.
what counts as a good idea
In 2015, I wrote a joke on Weibo:
The ML school is located in the mountains of the United States. Over the past century, martial arts geniuses have emerged in large numbers, and it has become the most famous school in the Jianghu. There are three sets of introductory martial arts in the school, namely: adding circles to graph models, adding layers to neural networks, and adding regularization to optimization goals. There are nursery rhymes as evidence: proficient in ML introductory skills, and can speak without writing.
In 2018, I continued with a short paragraph:
Within a few years, the DL God Sect in the north has sprung up. Internal training means learning, external neural network training, and many mental methods. They are called doors, attention, memory, confrontation, and enhancement. After the ImageNet battle, the martial arts world was shocked, and no one can get close to an Alpha dog. At that time, every family built alchemy furnaces, everyone was busy making alchemy, disciples gathered, and there were many dependents, and there was a tendency to dominate the world. There are nursery rhymes as evidence: big data in the left hand, NVIDIA in the right hand, and he will be busy making alchemy every peak.
The graph model plus circle, neural network plus layer, optimization target plus regularization mentioned here, the gates, attention, memory in the neural network, etc., are some innovative ideas to improve the performance of the model, which are widely used and published by major NLP tasks. Papers, perhaps due to the repeated use and publication of different NLP tasks, are somewhat aesthetically fatigued and lack deeper innovative ideas. Some netizens and scholars have criticized them as "watering", which seems to be not a good idea.
So what are good ideas? I understand that the word "good" has at least two meanings.
"Good" from the perspective of subject development
The essence of academic research is the exploration of unknown areas and the pursuit of answers to open questions. Therefore, from the perspective of promoting the development of disciplines, the criterion for judging what is a good research idea is first of all the word " new ".
In the past, there was a saying that the subject of artificial intelligence has a curse. Whenever artificial intelligence is solved (or has a solution), it is no longer considered to represent "human intelligence". The reason why computer vision, natural language processing, machine learning, and robots are still listed as the main directions of artificial intelligence may be precisely because they have not yet been solved and can still represent the dignity of "human intelligence". And if we want to carry out innovative research, we must come up with new ideas to solve these problems. The word "new" can be reflected in raising new problems and tasks, exploring new solutions, proposing new algorithm technologies, and implementing new tool systems.
On the basis of ensuring "newness", whether the research idea is good or not depends on how much it helps to promote the development of the discipline . The reason why deep learning has such a prominent influence is that it has had a revolutionary impact on various important directions such as artificial intelligence natural language processing, speech recognition, and computer vision, and has completely changed the understanding of unstructured signals (speech, image, Text) technical route of semantic representation.
"Good" from the perspective of research practice
Does that mean that as long as the idea is "new" enough? Is the newer the better? I think it should not be. Because, only ideas that can be made are eligible to be analyzed, okay? Therefore, from the perspective of research practice, the feasibility and verifiability of research ideas also need to be considered .
Realizability, which is reflected in whether the idea has enough mathematical or machine learning tools to support its realization. Verifiability is reflected in whether the idea has a suitable data set and widely accepted evaluation criteria. The reason why the ideas of many civilian scientists are not recognized by the academic community is because these ideas often lack realizability and verifiability, and only stay on paper, just some illusory ideas.
Where do good research ideas come from?
Whether an idea is good or not is not a black-and-white dichotomy, but a continuous distribution like a spectrum, which varies from time to time and from person to person. The development of the field of computer science and technology has both a process of accumulation and a singularity of transition. Accumulated quantitative changes will produce qualitative changes. The third steamed bun is full because of the first two steamed buns.
Today's academic research has become a highly specialized profession with a large group of researchers. "Publish or Perish" is something that people who are engaged in academic careers (such as professors, researchers, and graduate students) must do a good job of balancing. Researchers cannot be required to have every job of the "Nobel Prize" or "Turing Award" level It is worth publishing. As long as it contributes to the development of the research field, it is worth publishing to help peers move forward. Lu Xun said: Genius is not a monster that grows in the deep forest and wilderness by itself, but is produced and bred by the people who can make genius grow. Therefore, without such people, there will be no genius. This huge group of researchers is the mass basis for the growth of geniuses. At the same time, academic newcomers are constantly honing their ability to find good ideas in the process of carrying out innovative research training. Lu Xun also said: Even a genius, the first cry when he is born is the same as that of an ordinary child. good poem.
So, where do good research ideas come from? I conclude that first of all, you must have the ability to distinguish good research ideas from bad ones. This requires an in-depth and comprehensive understanding of the history and current situation of the research direction , specifically a comprehensive grasp of the subject literature. Humans are the best at learning animals, so they can completely take the ideas of research work in different periods in the existing literature as learning objects, and understand their impact on the development of disciplines after they are put forward—specifically reflected in paper citations, academic evaluations, etc.— —Establish an evaluation model for good and bad research ideas. It is difficult for us to analyze and perfectly list all the feature vectors that distinguish good and bad ideas, but the powerful learning ability of the human brain, as long as enough input data is given, it can automatically learn to establish a discriminative model in the neural network. Today, it is perhaps often referred to as academic insight.
Students who have done some research will feel that there will not be too many new ideas just by reading the literature in their own research direction. This is because what you read are thoughts from when the research problem has already been completed, they cannot in themselves inspire new ideas. How to generate new ideas? I conclude that there are three basic approaches that are feasible:
practice law . That is to implement the best existing algorithms on research tasks. By analyzing the experimental results, for example, it is found that these algorithms have extremely high computational complexity, the training convergence is extremely slow, or that the error samples of the algorithm show obvious patterns, which can inspire you. Ideas for improving existing algorithms. The latest algorithms on the Leaderboard of many natural language processing tasks now improve the algorithm in a targeted manner by analyzing error samples [1].
analogy . That is to establish an analogy between the research problem and other tasks, investigate the latest effective ideas, algorithms or tools on other similar tasks, and apply it to the current research problem through reasonable conversion and migration. For example, the attention mechanism was a great success in neural network machine translation. At that time, attention was mainly established at the word level. Later, Lin Yankai and Shen Shiqi of our research group proposed to establish sentence-level attention to solve the remote supervision training data of relationship extraction. Labeling the noise problem [2], this is an analogy.
combination method . That is to decompose the new research problem into several sub-problems that have been better solved, and establish a solution to the new research problem by organically combining the best practices on these sub-problems. For example, the pre-trained language model that we propose to integrate knowledge graphs is a new model built by combining existing algorithms such as BERT and TransE [3].
Just as the highest state in martial arts is to have no tricks to win with tricks, good research ideas are not limited to the above paths, and are often generated on the basis of researchers' deep understanding of research problems, comprehensive research experience and ingenuity. The result of "epiphany". It may be difficult for beginners to get a glimpse of the door. They need to start from the basic skills and go through a lot of scientific research and practical training before they can feel like entering the room.
In the process of scientific research practice, in addition to understanding history through reading a large number of documents, and generating insights through in-depth thinking and summarization, there is also an indispensable work, that is, active and open academic exchanges and cooperation awareness. The exchanges and collisions of ideas and achievements in different research fields not only provide new sources of innovative ideas, but also provide opportunities for "analogs" and "insights". Knowing the history, you can know that the proposal of artificial intelligence is the product of the interdisciplinary integration of mathematics, computer science, cybernetics, information theory, and brain science. The origin of the popular deep learning, Parallel Distributed Processing (PDP) in the 1980s, is also the product of the cooperation of researchers in computer science, brain cognitive science, psychology, biology and other fields. Below is the cover of the first volume of the famous book "Parallel Distributed Processing: Explorations in the Microstructure of Cognition" published in 1986.
The author talks about their collaboration process in the preface. In the first six months, they met twice a week to discuss research progress.
We expected the project to take about six months. We began in January 1982 by bringing a number of our colleagues together to form a discussion group on these topics. During the first six months we met twice weekly and laid the foundation for most of the work presented in these volumes.
The list of members of the PDP research group provided in the book still amazes me 40 years later because of its highly inter-institutional and interdisciplinary features. Therefore, it is especially recommended that students maintain an active awareness of academic exchanges on the premise of focusing on research issues during scientific research training. Whether it is listening to lecture reports, participating in academic conferences, or taking elective courses, consciously expand the breadth of academic exchanges. Not only mingle with small peers, but also academic partners in research fields that seem to be out of reach. With the enrichment of research experience, you will feel more and more strongly that the more long-span and intersecting academic reports, the more you will be inspired and generate more research ideas that excite you.
what beginners should do
Compared with reading papers, writing papers, designing experiments, etc., how to generate good research ideas is a link that is not well-defined, and it is difficult to summarize a fixed paradigm to follow. Like a pony crossing a river, you need to accumulate your own research experience through a lot of training and practice. However, for beginners, there are still several simple and feasible principles that can be referred to.
The publishable value of a paper depends on its Delta with the most directly related work . Most of our research work is advancing on the basis of predecessors' work. Newton said: If I can see farther than others, it is because I stand on the shoulders of giants. In my opinion, judging the value of a paper research idea is to see which giant's shoulders it stands on, and how far it has gone up on this basis. Conversely, before preparing to start a research work, when forming a research idea, it may be necessary to first clarify which giant's shoulders you are going to stand on and how you plan to go further. The Delta between the most directly related work determines how valuable the research idea is.
Take into account picking fruit and gnawing bones . People generally call the research idea that is easier to think of as Low Hanging Fruit (Low Hanging Fruit). Low-hanging fruits are easy to pick, but there are many people who pick them at the same time. If you choose to pick fruits, you will be easily troubled by ideas for loading. For example, in 2018, the pre-training language model headed by BERT made a major breakthrough, and in mid-2019, there were a lot of improvement work. Taking the cross-modal pre-training model as an example, more than Six pre-trained models for image-text fusion from different teams [4]. Putting yourself in the shoes of a person, conducting cross-modal pre-training model research is a direction that is relatively easy to think of. You must have the ability to predict, knowing that there will be many teams in the world who are also conducting research in this area at the same time. If you choose to enter field, it must be more in-depth and more distinctive, with its own unique contribution. Relatively speaking, there are few people who are willing to touch those difficult problems. It is also a good choice to concentrate on gnawing hard bones. Of course, at the same time, you will face the risk of not being able to solve them, or the risk of not getting too much attention if you solve them. . Students need to take into account two types of research ideas: picking fruit and gnawing bones according to their own characteristics, experience and needs.
Note the thematic coherence of multiple research efforts . The research training of students often lasts for several years, and it is necessary to pay attention to the theme coherence of multiple research works before and after to ensure the unity of internal logic. It needs to be considered that on the personal resume, in the personal statement for overseas application, or in various award presentations, these research results can be summarized together, and the general goals and general ideas of carrying out these research work can be stated. Objectively speaking, the pace of research in the field of artificial intelligence is fast, and the technology is updated quickly, so the publication of results also tends to be small, short and fast. I have friends from business schools and social sciences. Their research work often needs to last for one year or even more than several years; the research cycle of high-performance computing and computer network is relatively long. The characteristics of artificial intelligence such as small steps and fast running determine that many students will publish multiple papers even after graduating from undergraduate, not to mention master's and doctoral students. In this case, it is especially necessary to pay attention to the coherence and correspondence between the work before and after when researching the topic. When several research works are put together, whether they are separated from each other and cannot be said, or they are working hard for a unified goal, especially reflects the overall awareness and layout ability of the research. For example, the picture below is the chapter setting of Dr. Tu Cunchao's doctoral thesis "Network Representation Learning for Social Computing" when he graduated in 2018. On the whole, it is better than "Study on Some Important Issues of Social Computing" and other writing methods that have no internal connection. More convincing. Of course, for beginners, it is impossible to think clearly about the five-year research plan from the beginning. But thinking about it, or not thinking about it, the result is still different.
Pay attention to summarizing and grasping research dynamics and trends, and move with the times . In 2019, there was a question on Zhihu: "In the field of NLP in 2019, what valuable and promising work can individuals/teams with limited resources do?" My answer at that time was as follows:
I feel that the problems that the industry has begun to group together indicate that the main open problems have been almost solved, such as language recognition and face recognition, which have been widely used in commercial applications in the past 20 years. Looking at the recent BERT and GPT-2, I understand that it is more about maximizing the ability of deep learning to fit large-scale data. On the premise that the technical route of deep learning is basically mature, large companies have strong computing power to support it. More data can be used, the model can be made larger, and the effect fitting can be better.
When mature high-tech enters commercial competition, it will roughly conform to the development law of Moore's Law. Now training such as BERT seems out of reach, but with the development and popularization of computing power and other factors, maybe in a few years, everyone can easily train BERT and GPT-2, and everyone will be on the same starting line again. Eyes move on to the next challenging puzzle.
So it is better to consider in advance which problems cannot be solved by pure data-driven technology. Difficult tasks in NLP and AI, such as common sense and knowledge reasoning, complex context and cross-modal understanding, and explainable intelligence, have no feasible solutions, and I personally am not optimistic that data-driven methods can be completely solved. Higher-level cognitive abilities such as association, creation, and epiphany have not even been encountered. These are the directions that forward-thinking researchers should start focusing on.
It needs to be noted that research dynamics and trends are different in different periods. Capturing these dynamics and trends enables the development of results of interest to the research community. Otherwise, even if the research results have not changed, the results will be very different if the papers are simply submitted a few years earlier or later. For example, word2vec was published in 2013, and word representation learning research was carried out between 2014 and 2016, and it was relatively easy to be accepted by conferences such as ACL and EMNLP; Jobs are less common.
final addition
This short article is mainly aimed at beginners, introducing some experience and precautions in the process of seeking novelty, and I hope that everyone will avoid some detours. But reading the literature, thinking deeply, accepting the pain of rejecting manuscripts and continuously improving, you still have to eat. Academic research and the publication of papers may mean high salaries and scholarships for individuals, but their ultimate purpose is to truly promote the development of the discipline. Therefore, to do academic research that can stand the test, the key lies in "truth" and "newness", which requires us to always abide by and strive for. The famous historian and Tsinghua alumnus Mr. He Bingdi once mentioned in his autobiography "Sixty Years of Reading History and Reading the World" a sentence from the famous mathematician Lin Jiaqiao: "The most important thing is that no matter what line of work you are in, don't do second-class topics. . ” Specific to each field, what is the first-class topic itself is a matter of opinion, but it actually points to the inner "truth-seeking" attitude.
references
[1] The latest in Machine Learning | Papers With Code & Tracking Progress in Natural Language Processing | NLP-progress
[2] Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, Maosong Sun. Neural Relation Extraction with Selective Attention over Instances. The 54th Annual Meeting of the Association for Computational Linguistics (ACL 2016).
[3] Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, Qun Liu. ERNIE: Enhanced Language Representation with Informative Entities. The 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019).
[4] GitHub - thunlp/PLMpapers: Must-read Papers on pre-trained language models.