Anyone who engages in econometrics pays attention to this account
Manuscript: [email protected]
All the code programs, macro and micro databases and various software of the econometric circle methodology are placed in the community. Welcome to the econometric circle community for exchanges and visits.
Regarding the application of machine learning in metrological analysis, scholars can refer to the following articles: 1. The metrological regression module in Python and an overview of all modules, 2. A collection of spatial metrology software code resources (Matlab/R/Python/SAS/Stata), No longer feel lonely due to spatial effects, 3. Regression, classification and clustering: three directions to dissect the advantages and disadvantages of machine learning algorithms (implemented in Python and R), 4. Machine Learning Book One, Data Mining, Reasoning and Prediction , 5. From linear regression to machine learning, a picture to help you review the literature, a summary of 6.11 multivariate analysis methods related to machine learning, 7. machine learning and big data econometrics, you must read this, 8 . Machine learning and Econometrics book recommendations, worthy classics, 9. The latest trends in the application of machine learning in micrometrics: big data and causal inference, 10. The latest trends in the application of machine learning in micrometrics: regression models, 11. Machines The impact of learning on econometrics, an exclusive report at the AEA annual meeting, 12. Machine learning, which can be different from mathematical statistics, 13. The most complete collection of shortcuts in the history of Python, Stata, and R software! , 14. Comparison of Python and Stata, R, SAS, SQL in data processing, including code and detailed explanation, 15. Python for causal inference method examples, interpretation and code, 16. Text analysis steps, tools, methods and How to do visualization? 17. The application of text big data analysis in economics and finance, the most complete literature review, 18. Text functions and regular expressions, text analysis is no big deal, 19. The most complete: the application of deep learning in the field of economic and financial management Young and middle-aged scholars can't help but pay attention to the summary of current situation and frontier outlook 20.Top Frontier: Machine learning in agriculture and applied economics, its comparison with econometrics, you will be out if you don’t read it or not!
Earlier, we introduced ①"Machine learning methods appeared in top journals such as AER, JPE, QJE, etc.!", ②Frontier: Summary of application classification of machine learning in finance and energy economy, ③Lasso, ridge regression, elastic net estimation Interpretation of the implementation process and examples in the software", ④Depth analysis of regression methods (OLS, RIDGE, ENET, LASSO, SCAD, MCP, QR), ⑤High-dimensional regression methods: Are Ridge, Lasso, Elastic Net used, ⑥Lasso regression operation Guides, data, procedures and interpretation are available, ⑦Seven commonly used regression techniques, how to choose the regression model correctly?, ⑧Solutions for collinearity, excessive/unrecognized problems, ⑨Several recent developments and prospects of econometrics and experimental economics , ⑩New developments in econometrics, for reference, etc., aroused a huge response among academic colleagues.
Machine learning methods have gradually appeared in top journals in social sciences such as economic management, such as AER, JPE, QJE, JOF and other journals. In order to further understand the latest application trends of machine learning abroad, we present "Machine Learning: An Applied Econometric Method" to scholars. Scholars who are interested in machine learning methods are advised to study this very important and comprehensive article carefully. br /> ** ** text
on the bottom of the text content, author: Wei Zi Yi, Dongbei University of Finance and Economics, Communications E-mail: [email protected]
Machine Learning: an application of econometric methods
Mullainathan, Sendhil, and Jann Spiess. 2017. "Machine Learning: An Applied Econometric Approach."Journal of Economic Perspectives, 31 (2): 87-106.
摘要
Machines are increasingly doing "intelligent" things. Face recognition algorithms use a large dataset of photos labeled as having a face or not to estimate a function that predicts the presence y of a face from pixels x. This similarity to econometrics raises questions: How do these new empirical tools fit with what we know? As empirical economists, how can we use them? We present a way of thinking about machine learning that gives it its own place in the econometric toolbox. Machine learning not only provides new tools, it solves a different problem. Specifically, machine learning revolves around the problem of prediction, while many economic applications revolve around parameter estimation. So applying machine learning to economics requires finding relevant tasks. Machine learning algorithms are now technically easy to use: you can download convenient packages in R or Python. This also raises the risk that the algorithms are applied naively or their output is misinterpreted. We hope to make them conceptually easier to use by providing a crisper understanding of how these algorithms work, where they excel, and where they can stumble—and thus where they can be most usefully applied.
From the perspective of the effective use of machine learning in econometrics, the author of this document proposes that machine learning is not only a new tool in the econometrics toolbox, but supervised learning also solves the prediction problem: how to predict y from x. The advantage of machine learning is that it can discover generalized laws from data, and can discover complex structures that are not pre-specified. It can avoid simple overfitting, fit a complex and flexible model from the data, and make the model run well outside the sample. At the same time, the article also makes a detailed summary of the working principle of machine learning and its combined application with econometrics. The integration of the two will help us expand the scope and depth of our research.
The working principle of machine learning
First, the author compares machine learning with the familiar OLS method in measurement by using an example of predicting the value of a house. Select 10,000 houses randomly selected from the metropolitan sample from the 2011 U.S. Housing Survey as the training sample, and select information about the house and its location (such as the number of rooms, basic area, etc.) as variables. A total of 150 variables are selected. For different prediction methods, the author analyzed the house value prediction effect of 41,808 retained samples separated from the same sample by evaluating each method. The results are shown in the following table: 
Note: For all the detailed information about the sample empirical research, you can refer to: http://e-jep.org .
Two findings in the above table:
- Emphasizes the necessity of selecting retained samples for evaluation. For some machine learning algorithms (such as random forest), this algorithm is particularly prone to overfitting.
- In the evaluation of retained samples, even when the sample size is moderate and the number of covariates is limited, machine learning algorithms such as random forests can do better than ordinary least squares.
One. From linear least squares to regression tree.
In the above problem, for ordinary least squares regression, when considering the interaction between variables (such as the increased value of the fireplace may vary with the number of living rooms), we need to manually The calculation determines which interactions are included in the regression (because if you include all the regression variables with more paired terms than data points). When the problem changes from house value to a more complex face recognition problem, the function of effectively combining pixels will be highly non-linear and interactive, and the feasibility of using ordinary least squares regression will be greatly reduced.
Furthermore, the author uses regression tree as an example to describe the automatic search of machine learning for interaction. As shown in the figure below, the regression tree maps each vector of house features to predicted values. The prediction function takes the form of a tree, which is divided into two at each node. At each node of the tree, the value of a single variable (such as the number of toilets) determines whether to consider the child node on the left (less than two toilets) or the child node on the right (two or more). The predicted value is returned when it reaches the most terminal, the leaf, and each leaf corresponds to the product of the dummy variable (such as the leftmost picture, the coefficient value is 9.2).
two. Overfitting and its solutions
Consider, if there is a tree deep enough, then each observation will stay in its own leaves. Then for the given sample set, this will be a perfect fit. But at the same time this is also perfect overfitting.
From the above figure, we can see that for the leftmost figure, there is a certain distance between the estimated value and the true value, and the fitting effect is not satisfactory. As for the graph on the far right, although each sample point is on the curve, because the function form is too complicated, it is not possible to have a good estimate for the points outside the sample point, and overfitting occurs. Case.
The biggest attraction of machine learning lies in its high dimensions: flexible functional forms allow us to adapt to various structures of data. But this flexibility also offers so many possibilities that simply choosing the most suitable function for the sample would be a bad choice. So how does machine learning perform out-of-sample predictions?
The first part of the solution is regularization. Taking regression trees as an example, we can choose the best tree among trees with a certain depth instead of choosing the "best" tree among all trees. The shallower the tree, the worse the fit within the sample: there are many observations on each leaf, and none of the observations can fit well, but this also means that the degree of overfitting is low. The depth of the tree is an example of a regularizer, which can measure the complexity of the function, and by appropriately selecting the regularization level, the occurrence of overfitting can be avoided.
So how to choose the level of regularization? That involves the second part of experience adjustment. The essence of overfitting is that we want the prediction function to perform as well outside the sample as inside the sample. Through empirical adjustment, an out-of-sample experiment was created in the original sample. Fitting a part of the data and seeing which level of regularization can make another part of the data get the best performance. In addition, the efficiency of this process can be improved by cross-checking: the sample is divided into K sub-samples of equal size. A single subsample is retained as the data for the verification model, and the other K-1 samples are used for training. The cross-validation is repeated K times, and each sub-sample is validated once. Finally, we select the parameter with the best estimated average performance.
Further, the author summarizes the model type F and its regularizer R(f) of different prediction algorithms.
Choosing a prediction function involves two steps:
taking complexity as a condition, choosing the best in-sample minimum loss function.
Use empirical adjustments to estimate the optimal level of complexity.
three. The guiding role of econometrics
therefore requires consideration and choice in how to choose the appropriate function and how to regularize it when using machine learning methods. The results of many recent econometric theories have supplemented the theory of machine learning, revealing the comparative performance of different regularization tools.
For example, for the area of the room, should only consider the total area or the area of each room, should logarithmic processing be used for variables, or normalization, and so on. These choices about how to present the elements will make the model type and the regularizer affect each other: the linear model can easily get the area of each room from the area and the number of rooms, while the regression tree requires many split nodes. In a traditional forecasting model, changing the form of a set of variables alone does not change the forecast, because the type of model selected from it does not change. However, at any given level of regularization, the model type may change, leading to changes in the predicted value. For example, if we think that the number of bathrooms per bedroom is important in the pricing process, then creating this variable will significantly reduce complexity costs.
At the same time, a reliable assessment of predictive performance is noncommittal, and this also requires a strong guarantee of econometrics. In the housing price example, dividing the sample into training samples and retention samples conforms to the principle of econometrics: all data involved in fitting the prediction function (including cross-validation used to verify the algorithm) are not used to evaluate the generated prediction function.
Therefore, econometric theory plays a dual role here. First, econometrics can guide design choices, such as cross-checking fold selection and model types. These selection guidelines can help improve the quality of predictions and any test functions based on them. Second, for a given fitted prediction function, it must be able to infer the estimated degree of fit. The retention of samples allows us to test against the predicted value of the fitted function.
four. The disadvantages of
machine learning The advantage of machine learning is that they can fit many different models. But this also leads to a fatal weakness: more models means that two models with completely different parameters can produce similar prediction results. So how the algorithm chooses between two completely different functions boils down to a coin toss problem. Therefore, how we choose between two different models is worth studying.
Regularization also exacerbates this problem. First of all, we will choose a relatively uncomplicated but wrong model; second, it can cause deviations of missing variables, and when regularization excludes some variables, it may lead to deviations in parameter estimates.
How to apply machine learning
Through the above, we can know that the advantage of machine learning is that it provides a powerful, flexible and high-quality prediction method, but its weakness is that it is difficult to verify the hypothesis, because machine learning does not produce the stability of the basic parameters. estimate. Therefore, machine learning has great application value in the improvement and prediction of pictures.
1. New data
nowadays "big data" has become a hot word. On the one hand, it emphasizes changes in the scale of data, and on the other hand, the nature of these data has also undergone an equally important change. Machine learning can handle high-dimensional unconventional data that is difficult for standard estimation methods, including images and language information that are traditionally not even considered usable data. The relevant literature is organized as follows:
2. Estimation and prediction The
second type of application is the task of estimation and prediction. For example, to understand the two-stage regression process in the case of linear instrumental variables:
first perform regression on the instrumental variable z:
- Then regression on the fitted value picture
usually takes the first stage as the estimation step. But this is actually a forecasting task: only forecasting enters the second stage. The coefficients in the first stage are just one way to reach these fitted values. The limited sample deviation in instrumental variables is the result of overfitting. Overfitting means that the picture of fitted values within the sample not only picks up pictures, but also picks up noisy pictures. As a result, the picture is biased towards x, so that the estimated value picture of the second stage is therefore biased towards the ordinary least squares estimation of x and y. Relevant documents are organized as follows:
3. The
related literature on policy prediction is as follows:
4. The
related literature on test theory is as follows:
Conclusion
For empiricists, theory-driven and data-driven analysis modes always coexist. Theory-driven evaluation methods are mostly based on top-down theoretical deductive reasoning estimation models. For example, the application of economics mostly revolves around parameter estimation: good estimation of parameters that depend on and explain the relationship between variables. Most data-driven evaluation methods simply let the data speak. Machine learning provides a powerful tool to hear the content of the data more clearly. Different from parameter estimation, machine learning is part of the forecasting toolbox in econometrics and is more suitable for forecasting tasks in economics. These two methods do not conflict. Theory can guide which variables to manipulate in experiments; but when analyzing results, machine learning can help manage multiple results and estimate the effects of heterogeneous treatments.
In the long run, new empirical tools can help expand the types of problems we are studying. Ultimately, machine learning tools may expand the scope of our work, not only by providing new data or new methods, but also by allowing us to focus on new problems.
Long press the above QR code to read the original text of Machine Learning: An Applied Econometric Approach
For a compilation of some measurement methods, scholars can refer to the following articles: ① "200 articles used in empirical research, a toolkit for social science scholars", ② 50 famous experience posts commonly used in empirical article writing, a must-read series for students ③The Articles album on Chinese topics on the AER in the past 10 years. ④AEA announced the top ten research topics that received the most attention in 2017-19, giving you the direction of the topic selection. ⑤The key topic selection direction of the top Chinese journals in 2020, just write the paper These, ⑥The road map of "high light moments" in the past 30 years, RCT, DID, RDD, LE, ML, DSGE and other methods. Later, we introduced a collection of selected articles using CFPS, CHFS, CHNS data for empirical research! , ②These 40 micro-databases are enough for your Ph.D., anyway, relying on these libraries to become a professor, ③The most complete collection of shortcut keys in the history of Python, Stata, and R software! , ④ 100 selected Articles albums about (fuzzy) breakpoint regression design! , ⑤ 32 selected Articles of DID about the double difference method! , ⑥ 33 selected Articles of SCM about the synthesis control method! ⑦Compilation of the latest 80 papers about China's international trade field! ⑧Compilation of 70 recent economic papers on China's environmental ecology! ⑨A collection of selected articles using CEPS, CHARLS, CGSS, CLHLS database empirical research! ⑩Compilation of the last 50 papers using the system GMM to conduct empirical research!
The following short-linked articles belong to a collection, you can collect them and read them, or you won't find them in the future.
In 2.5 years, nearly 1,000 non-weighted measurement articles in the econometric circle,
You can search for any measurement related issues directly in the official account menu bar,
Econometrics Circle