Econometric results are often used to support policy decisions, academic research, or business strategies. But how can you be sure that your results are reliable, robust, and valid? In this article, you will learn some basic steps to test the strength of your econometric results and avoid common pitfalls.
The first step is to choose the appropriate econometric methods for your research question, data, and assumptions. For example, if you want to estimate the causal effect of a treatment on an outcome, you need to use methods that account for endogeneity, selection bias, or omitted variables. If you want to analyze time series data, you need to check for stationarity, cointegration, and autocorrelation. If you want to compare different groups, you need to control for heterogeneity, confounding factors, and interaction effects. Choosing the wrong methods can lead to biased, inconsistent, or inefficient estimates.
The bottom line of all research with data is that you do not want to compare apples with oranges. The golden standard to solve this apples and oranges problem is to design a randomized controlled trial (a topic on its own). But what if you cannot run a randomized controlled trial? Suppose you just have a dataset on customer sales (y-variable) and advertising expenditures (x-variable) in different regions. The best hands-on way to start is to compare the characteristics of the high-advertising regions with the low-advertising regions. If they are very different in many dimensions, this is an early warning sign that you are not comparing apples with oranges. You need more advanced models, such as matching, to answer your question.
The second step is to check the model diagnostics to assess the quality of your estimates and the validity of your assumptions. For example, you can use tests for multicollinearity, heteroskedasticity, autocorrelation, normality, specification, or structural breaks. You can also use graphs to visualize the distribution, relationship, or residuals of your variables. Checking the model diagnostics can help you identify and correct potential problems that can affect the accuracy, precision, or inference of your results.
The third step is to perform sensitivity analysis to examine how your results change under different scenarios, assumptions, or specifications. For example, you can use alternative measures, data sources, or samples to test the robustness of your results. You can also use different functional forms, control variables, or estimation methods to test the sensitivity of your results. You can also use simulation, bootstrap, or jackknife techniques to test the stability of your results. Performing sensitivity analysis can help you measure the uncertainty, variability, or generalizability of your results.
The fourth step is to compare your results with the existing literature and explain the similarities or differences. For example, you can review previous studies that have addressed the same or related research questions, used the same or similar data sets, or applied the same or comparable methods. You can also discuss how your results contribute to the existing knowledge, fill the gaps, or challenge the findings. Comparing with the existing literature can help you contextualize, validate, or improve your results.
This should be second step; it will save you time and give you insightful information on the existing knowledge if you review the literature. This will help you understand the existing consensus and the differences in the past studies that may influence the model and the data you use for your research. Essentially, you want to know what is the latest information on this particular research topic before you expand the frontier of knowledge.
The fifth step is to report your results clearly and transparently, following the standards and conventions of your field. For example, you can use tables, figures, or text to present your results in a concise, consistent, and comprehensible way. You can also use appropriate notation, symbols, or labels to identify your variables, coefficients, or statistics. You can also provide information on the data sources, methods, assumptions, tests, or limitations of your analysis. Reporting your results clearly can help you communicate, interpret, or replicate your results.
The final step is to seek feedback from peers, experts, or reviewers on your results and your analysis. For example, you can ask for comments, suggestions, or questions on your methods, data, results, or interpretation. You can also request for verification, validation, or replication of your results by sharing your data, code, or documentation. You can also acknowledge and address the feedback in a constructive, respectful, and responsive way. Seeking feedback can help you improve, refine, or confirm your results.
Economic significance Economic significance refers to the magnitude of a result in terms of its economic impact. It tells us how important a result is. The economic significance of a coefficient can be tested by multiplying the coefficient by the standard deviation of the variable: Change independent variable = Estimated coefficient * Standard deviation of the independent variable. A coefficient is considered to be economically significant if it is large enough to have a meaningful impact on the dependent variable. To determine whether a coefficient is meaningful, you could compare the estimated change in the dependent variable with the actual mean and standard deviation of the dependent variable.
Robustness checks You can check the robustness of your econometric analysis by measuring how well your model fits the data you are analyzing. This can also be done by performing robustness checks which come standard with a number of statistical analysis software including R, STATA, Matlab, just to mention a few.