Reprinted: https://blog.csdn.net/godloveyuxu/article/details/66969715
Original: http://codecapsule.com/2012/01/18/how-to-implement-a-paper/
Author: Emmanuel Goossaert
This article is a short guide to implementing algorithms from scientific papers. I have implemented many complex algorithms from books and scientific publications. This article summarizes what I have learned in searching, reading, coding and debugging. This is obviously limited to publications related to the field of computer science. However, you should be able to apply the guidelines and good practices provided below to any type of document or implementation.
1-before you enter
Before you read the technical paper and implement it, you should check a few points. Whenever you are ready to start such a project, make sure you cover them carefully.
1.1-Find an open source implementation to avoid coding
Unless you want to execute the thesis in order to learn more about this field, you don't need to implement it. In fact, what you want is not to encode the paper, but to implement the code of the paper. So before you start anything, you should spend a few days trying to find an open source implementation on the Internet. Just think about it: Would you rather lose two days of searching for code, or waste two months to implement an already usable algorithm?
1.2-Find an easier way to achieve your goals
Ask yourself what you want to do and if a simpler solution can meet your needs. You can use other techniques-even if the result is only 80% of what you want-there is no need to implement the document, and you can run the available open source libraries in the next two days? For more information, see my article 20/80 Productivity Rules .
1.3-Beware of software patents
If you are in the United States, please pay attention to software patents. Some papers are patented, and you may have trouble using them in commercial applications.
1.4-Learn more about the field
If you are reading an article about using support vector machines (SVM) in the context of computational neuroscience, then you should read a short introduction to machine learning and the different types of classifiers that can replace SVM, you should read about computational neuroscience General articles to understand what is currently being studied.
1.5-Stay motivated
If you have never performed paper and/or if you are new to the paper field, then reading can be very difficult. No matter what happens, don't let the number and complexity of mathematical equations stop you. In addition, speed is not an issue: even if you feel that your understanding of the paper is slower than you want, as long as you continue to work, you will see that you will slowly and steadily understand the concepts presented in the paper and pass all the difficulties one by one .
2-Three types of papers
It is a good choice to choose a random paper and start execution immediately. There are a lot of papers there, which means there is a lot of rubbish. All publications can be divided into three categories:
2.1-Groundbreaking paper
Some really interesting, well-written and original research. Most of these papers are from top universities, or from research teams at small universities that have dealt with this problem for about six to ten years. It was easy to find later: they cited their publications in their papers, indicating that they had been working on this issue for some time, and their new work was based on the successful record of the publication. In addition, these groundbreaking papers are generally published in the best journals on site.
2.2-Template paper
Some research groups are following the work of pioneering teams, suggesting improvements, and publishing their improvements. Many of these papers lack correct statistical analysis and mistakenly conclude that the improvement is really breaking the original algorithm. Most of the time, they really didn't bring anything except for unnecessary extra complexity. But not all imitators are bad. Some are good, but rare.
2.3-Trash paper
Some researchers really don't know what they are doing and/or evil. They are just trying to maintain their status and privileges in the academic institutions they teach. So they need funds, so they need to publish, something. Honest people will tell you in the conclusion that they failed and the result is only accurate N% of the time (N is bad value). But some evil people will lie, and their research has achieved great success. After reading the publication for a period of time, it is easy to find trash paper and stain it.
3-How to read scientific papers
Many have already written on this topic, so I won't write too much. A good starting point is how to read Srinivasan Keshav's paper . Here are a few points I found useful when reading scientific publications.
3.1-Find the correct file
What you want to achieve is an original paper, a paper that starts the entire field. Sometimes, if you feel that it brings real improvement and consistency to a good but immature seminal paper, then you can choose a template paper.
So let us say that you have a piece of paper as your starting point. You need to do some research around it. To this end, the strategy is to find relevant publications, as well as those listed in the "References" section at the end of this article. Go to Google Scholar and search for the title and author. Is any paper you found better than the original paper? If so, throw away the paper you are looking at first, and then keep the new paper you found. Another cool feature of GoogleScholar is that you can find papers that cite a given paper. This is really great, because all you have to do is follow the citation chain from one paper to the next and you will find the most recent paper. Finding a good paper from the starting point is that all the papers on the search are cited by the current paper, and the paper cited the current paper. Move in time, and you should find high-quality paper that meets your needs.
Important note: At this simple exploration and calculation stage, you should not read and fully understand the paper. This search for the correct article should only be done by copying the document and using your instinct to detect junk (this comes with experience).
3.2-Don't read on the screen
Print the publication on hard paper and read the paper version. Also, do not reduce the size in order to print more on each page. Yes, you will save three sheets of paper, but you will lose time because you will read these small characters quickly. The font size for good reading is between 11 and 13 points.
3.3-Good time and location
Don't look at the paper in the middle of the night, do it at the moment of the day when the brain is still fresh. Also, find a quiet area and use good lighting. When I read, I have a desk lamp that points directly to the file.
3.4-Marks and notes
Use markers to highlight important information and take notes on the edge of any thoughts that pop up in your head as you read.
3.5-Understand the definition of all terms
When you are used to reading most news articles and novels, your brain is well-trained to fill in the meaning of words you don't know by using context as a deduction device. Reading scientific publications is a different exercise, and one of the biggest mistakes is assuming the wrong meaning of a word. For example, in this sentence "the result of this segmentation method still suffers from fuzzy artifacts". These two words, "segment" and "cultural relics", have general meanings in English, but they also have specific meanings in the field of computer vision. If you don’t know that these words have a specific meaning in this article, then you don’t pay attention when reading, your brain will fill in the general meaning and you may lose some very important information. Therefore, you must (i) avoid assumptions about the word, and whenever in doubt, look for the word in the context of the publication’s field, and (ii) write a piece of paper A glossary of all concepts and vocabulary specific to the publication. If you encounter concepts such as "realistic point" and "piecewise affine transformation" for the first time, then you should look up its precise definition and write it into your vocabulary. Concepts are brain shortcuts that support language, allowing you to understand the author’s intentions faster. If you encounter concepts such as "realistic point" and "piecewise affine transformation" for the first time, then you should look up its precise definition and write it into your vocabulary. Concepts are brain shortcuts that support language, allowing you to understand the author’s intentions faster. If you encounter concepts such as "realistic point" and "piecewise affine transformation" for the first time, then you should look up their precise definitions and write them into your vocabulary. Concepts are brain shortcuts that support language, allowing you to understand the author’s intentions faster.
3.6-Look for statistical analysis in the conclusion
If the author only provides a curve from his algorithm, and extracts a curve from another algorithm, and says "Look, it is 20% more accurate", then you know you are reading garbage. What you want to read is: "In a set of tests of N instances, our algorithm shows a significant improvement with a p-value of 5% using a two-sample t-test." Using statistical analysis shows that the author, and is A good proof that the results can be trusted for generalization (unless the author lied about making their results look sexier, which can happen forever).
3.7-Make sure the conclusion proves that this article is doing what you need
Suppose you want an algorithm that can find any face in a picture. The author of this article stated in the conclusion that their model was trained using 10 poses from 80 different people (10x 80 = 800 pictures). The accuracy of face detection in the training group was 98%, but only 70% compared with the test. Set (pictures were not used during training). What does it mean? This means that, obviously, the algorithm has a proper generalization problem. It performs well when used on the training set (useless), and performs worse when used in the real world. At this point you should conclude that maybe this article is not good enough for what you need.
3.8-Pay attention to the input data used by the author
If you want to perform face detection using a webcam, and the author has used a photo taken with a high-definition camera, then in your case, the algorithm will not perform as the author did. Make sure that the algorithm is tested on data similar to yours, otherwise you will end up with a good implementation that is completely unusable in actual settings.
3.9-The author is human
The authors are human, so they make mistakes. Don’t think that the author is absolutely correct. If an equation is really difficult to understand or follow, you should ask yourself if the author made a mistake there. This may just be a typo in the text, or a mathematical error. In either case, the best way is to formulate the equation yourself and try to verify the results.
3.10-Understanding variables and operators
The main task in the publication implementation process is to convert the mathematical equations in the papers into codes and data. This means that before jumping into the code, you must understand 100% of these equations and processes. For example, "C= A". B "may have different meanings. A and B can be simple numbers,". "The operator can simply become a product. In this case, C will be the product of two numbers A and B. But it is also possible that A and B are matrices,". "Represents the matrix product operator. In this case, C will be the product matrix of matrices A and B. Another possibility is that A and B are matrices, ". "Is the term "item-wise" product operator. In this case, each element C(i,j) is the product of A(i,j) and B(i,j). The symbols of variables and operators can be Change from one mathematical formula to another, from one research group to another. Make sure you know what each variable is (scalar, vector, matrix, or something) and what each operator does with those variables.
3.11-Understanding the data flow
A paper is a series of equations. Before you start coding, you must know how to insert the output of equation N into the input of equation N+1.
4-Prototyping
Once you have read and understood this article, then it is time to create a prototype. This is a very important step and avoiding it can lead to a waste of time and resources. Implementing complex algorithms in languages such as C, C++ or Java can be very time consuming. Even if you have some confidence in the paper and think that the algorithm will work, there is still a chance that it will not work at all. So you want to be able to encode as fast as possible in the dirtiest way, just to check if it works.
4.1-Prototyping solution
The best solution is to use a higher-level common language or environment such as Matlab, R, Octave or SciPy/NumPy. It is not easy to express a mathematical equation in C++ and then print the result to manually check it. On the contrary, writing equations in Matlab is very simple, and then print them out. It takes two to three weeks in C++, and you will spend two days in Matlab.
4.2-Prototype development helps the debugging process
One advantage of having a prototype is that when you have a C++ version, you can debug by comparing the results between the Matlab prototype and the C++ implementation. This will be further developed in the "Debugging" section below.
4.3-Clear execution problems beforehand
You will definitely cause software design errors in your prototype, which is a good thing because you will be able to determine the difficulties of the process or data. When you write the C++ version, you will understand how to build the software better, and you will generate more concise and stable code than there is no prototyping step (this is the idea of "throwing away the system" proposed by Frederick Brooks in Mythology Month ).
4.4-Verify the results presented in the article
Read the "experiment" part of this article carefully and try to reproduce the experimental conditions as closely as possible by using the test data used by the author as closely as possible. This increases your chances of rewriting the results obtained by the author. Failure to use similar conditions may result in actions that you perform, which you may regard as errors, and you just fail to provide correct data. Once you can reproduce the results based on similar data, you can start testing on different types of data.
5-Choose the right language and library
At this stage, you must have a clear understanding of the algorithms and concepts provided in the publication, and you must have a working prototype to convince the algorithm that the algorithm is actually processing the input data you wish to use in production. Now is the time to move on to the next step, which includes implementing the publication in the language and framework you wish to use in production.
5.1-Pre-existing system
In many cases, production languages and libraries are determined by existing systems. For example, you have a set of algorithms for normalization of lighting in pictures, in a library coded in Java, and you want to add new algorithms from the release. In this case, obviously, you will not write this new algorithm in C++, but in Java.
5.2-Predict the implementation of future use
In this case, there is no pre-existing system to impose a language, so the choice of language should be based on the predicted use of the algorithm. For example, if you think that within four to six months, the possible port of your application will be completed to the iPhone, then you should choose C/C++ over Java, because it will be the only way to easily integrate the code into the target Method-C application, no need to start from scratch.
5.3-Available libraries for fully or partially solving algorithms
Available libraries in different languages can also target the choice of production languages. We assume that the algorithm you wish to implement uses well-known algebraic techniques such as principal component analysis (PCA) and singular value decomposition (SVD). Then you can write PCA and SVD from scratch. If there is an error, it may lead to the end of a week of debugging, or you can reuse the libraries that have implemented these technologies, and use conventions and Matrix to write the library's classes. Ideally, you should be able to decompose your implementation into subtasks and try to find libraries that implement as many of these subtasks as possible. If you find a perfect library for only a certain language, then you should choose that language. Also, please note that the choice of library should be a trade-off between reusing existing code and minimizing dependencies. Yes, it is good to write code for each subtask required for the implementation, but if you need to create dependencies for 20 different libraries, then this may not be very practical and may even endanger the future stability of the implementation.
6-Implementation
Here are some tips on my experience in implementing publications
6.1-Choose the right precision
You should carefully choose the type of calculation you use. It is usually better to use double instead of float. The memory usage can be larger, but the accuracy of the calculation will be greatly improved, which is generally worthwhile. In addition, you should understand the difference between 32-bit and 64-bit systems. Whenever, create your own type to encapsulate the underlying type (float or double, 32-bit or 64-bit) and use this type in your code. This can be done by defining a class in C/C++ or Java.
6.2-record everything
Although excessive documentation can greatly slow down the authenticity of the project, in the case of complex technical documentation, you need to comment on everything. Even if you are the only person working on the project, you should document the documents, courses and methods. Choose a routine, such as Doxygen or reStructuredText, and stick to it. During the development process, there will be a period of time when you will forget how some classes work, or how you implement some methods, and you will thank you for writing your own code!
6.3-Add a reference to this article in the code
For each equation in the paper you implement, you need to add a comment that cited the article (author and year) and the paragraph number or equation number. This way, when you re-read the code later, you will be able to connect the code directly to the precise location in the paper. These comments should be:
// See Cootes et al.， 2001， Equation 2.3
// See Matthews and Baker， 2004， Section 4.1.2
6.4-Avoid using mathematical symbols in variable names
Suppose some quantities in the algorithm are a matrix denoted as A. Later, the algorithm requires the gradient of the matrix in two dimensions, expressed as dA=(dA/ dx, dA/ dy). Then the names of the variables should not be "dA_dx" and "dA_dy", but "gradient_x" and "gradient_y". Similarly, if the equation system requires convergence testing, the variables should not be "prev_dA_dx" and "dA_dx", but "error_previous" and "error_current". Always name the physical quantities they represent, rather than any alphabetic symbols used by the author of the article (for example, "gradient_x" instead of "dA_dx"), and always indicate a less specific specific from left to right (for example, "gradient_x" and Not "x_gradient").
6.5-Don't optimize during the first pass
Leave all optimizations for later. Because you can never determine which part of the code needs to be optimized. Every time you see a possible optimization, add a comment and explain how to achieve the optimization, such as:
// OPTIMIZE HERE:computing the matrix one column at a time
// and multiplying them directly could save memory
This way, you can find all the places in the code that can be optimized and get new tips on how to optimize. Once your implementation is complete, you will be able to find the location to optimize by running Profiler (such as Valgrind) or any availability in your programming language.
6.6-Planning to create an API?
If you plan to use current code as the basis for an API that grows over time, then you should understand the techniques for creating actually usable interfaces. To this end, I would recommend "compiled library" technology, summed up by the JoshuaBloch in his speech, such as how to design a good API and why it is important.
7-Debug
Implementing a new algorithm is like cooking dishes you have never eaten. Even if it tastes good, you will never know what it tastes like. Now we are lucky, because unlike cooking, software development has some useful techniques to increase our confidence in implementation.
7.1-Compare results with other implementations
A good way to clear errors is to compare the results of the code with the results of existing implementations of the same algorithm. Because I assume that you have correctly completed all the tasks mentioned above "but before the jump", you did not find any available algorithm implementation (otherwise you would use it instead of implementing this paper!). Therefore, your only other implementation at this stage is the prototype you programmed earlier.
Therefore, the idea is to compare the results of the prototype and the production implementation at each step of the algorithm. If the results are different, then one of the two implementations will go wrong and you must find which and why. The accuracy can be changed (the prototype can give you x = 1.8966 and production code x = 1.8965), and the comparison should of course take this into account.
7.2-Talk to the person reading the paper
Once all steps of the two implementations (prototype and production) get exactly the same results, you can gain some confidence that your code is bug-free. However, you still have a risk that you misunderstood the paper. In this case, these two implementations will provide the same result for each step, and you will think your implementation is good, and this just proves that both implementations are equally wrong. Unfortunately, I have no way to know this kind of problem. Your best bet is to find someone who has already read the paper and ask questions about the part of the algorithm you are not sure about. You can even try to ask the author, but your chances of getting an answer are low.
7.3-Visualize your variables
In the development process, always pay attention to the content of the variables used by the algorithm. I'm not talking about just printing all the values in the matrix and data, but finding that visualization techniques are applicable to any variable in the implementation. For example, if you assume that the matrix represents the gradient of an image, during encoding and debugging, you should have a window pop up and display the gradient image, not just the numeric values in the image matrix. This way, you will associate the actual image with the data you are processing, and you will be able to detect when a problem with one of the variables occurs, which in turn will indicate possible errors. The visualization techniques invented include images, scatter plots, graphs or anything that is not just a silly list1,
7.4-Test data set
Generating data to experiment with your implementation can be very time consuming. At any time, you can try to find a database (face database, text extraction database, etc.) or tools for generating such data. If not, don't manually generate 1000 samples. Write a fast data generator in 20 lines and complete it.
in conclusion
In this article, I propose good practices for implementing scientific publications. Remember, these are only based on my personal experience and should not be followed blindly. When reading and writing, please pay attention and use your judgment to determine that the above guidelines are suitable for your project. Maybe some practice will hurt your project more than it will help it, it depends on you to find out. Now go and implement some cool algorithms!