Simple and complex difference-in-differences approach
DOI:
https://doi.org/10.22394/1993-7601-2024-73-119-142Keywords:
метод разности разностей, условие параллельных трендов, непоследовательное воздействие, вероятное воздействие, квантильный эффект воздействияAbstract
The paper presents extensions of the popular difference‐in‐differences approach (DD) from 2×2 design on multiple time‐period, multiple groups, fuzzy DD, non‐staggered treatment and approaches to measure distributional treatment effect. The paper describes assumptions for consistent estimation of the treatment effect by two‐way fixed effects model (TWFE) and presents the problem leading to inconsistent estimates justifying the application of alternative estimators. The paper briefly introduces methods developing DD for multiple‐period multiple‐group cases based on TWFE and alternative approaches. The proposed techniques allow treatment evaluation in the frame of DD when canonical TWFE leads to inconsistent estimates. Some approaches allow replacement of the well‐known parallel trend assumption (PTA) for a conditional PTA or time randomisation. The paper refers to implementations of these methods in Stata and R. Simulation modelling demonstrates that the stated properties of the alternative estimators are not always reliable.Agencies
The publication was prepared as a result of research conducted under Project No. 23-00-033 'Assessing the Impact of Macro Shocks on Socio-Economic Processes in the Regions of Russia (Using COVID-19 as an Example)' within the framework of the Program 'Scientific Fund of the National Research University Higher School of Economics (HSE University)'Downloads
References
Вулдридж Дж. М. (2009). Оценивание методом «разность разностей». Квантиль, 6, 25–47.
Ashenfelter O. (1978). Estimating the effect of training programs on earnings. Review of Economics and Statistics, 60 (1), 47–57.
Ashenfelter O., Card D. (1985). Using the longitudinal structure of earnings to estimate the effect of training programs. Review of Economics and Statistics, 67, 648–660.
Athey S., Imbens G. W. (2006). Identification and inference in nonlinear difference-in-differences models. Econometrica, 74 (2), 431–497. DOI: 10.1111/j.1468-0262.2006.00668.x.
Bertrand M., Duflo E., Mullainathan S. (2004). How much should we trust differences-in-differences estimates? Quarterly Journal of Economics, 119 (1), 249–275. DOI: 10.1162/003355304772839588.
Bilinski A., Hatfield L. A. (2018). Seeking evidence of absence: Reconsidering tests of model assumptions. ArXiv:1805.03273. DOI: 10.48550/arXiv.1805.03273.
Bonhomme S., Sauder U. (2011). Recovering distributions in difference-in-differences models: A comparison of selective and comprehensive schooling. Review of Economics and Statistics, 93 (2), 479–494. DOI: 10.1162/REST_a_00164.
Borusyak K., Jaravel X. (2018). Revisiting event study designs with an application to the estimation of the marginal propensity to consume. https://scholar.harvard.edu/files/borusyak/files/borusyak_jaravel_event_studies.pdf.
Borusyak K., Jaravel X., Spiess J. (2021). Revisiting event study designs: Robust and efficient estimation. ArXiv:2108.12419. DOI: 10.48550/arXiv.2108.12419.
Butts K. (2021). did2s: Two-stage difference-in-differences following Gardner (2021). R package version 1.0.2. https://cran.r-project.org/web/packages/did2s/vignettes/Two-Stage-Difference-in-Differences.html.
Callaway B., Sant’Anna P. H. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225, 200–230. DOI: 10.1016/j.jeconom.2020.12.001.
Callaway B., Li T. (2019). Quantile treatment effects in difference in differences models with panel data. Quantitative Economics, 10 (4), 1579–1618. DOI: 10.3982/QE935.
De Chaisemartin C., D’Haultfoeuille X. (2018). Fuzzy differences-in-differences. The Review of Economic Studies, 85 (2), 999–1028. DOI: 10.1093/restud/rdx049.
De Chaisemartin C., D’Haultfoeuille X. (2020). Two-way fixed effects estimators with heterogeneous treatment effects. American Economic Review, 110 (9), 2964–2996.
De Chaisemartin C., D’Haultfoeuille X. (2021). Difference-in-differences estimators of intertemporal treatment effects. NBER Working Paper No. 29873.
De Chaisemartin C., D’Haultfoeuille X. (2022). Two-way fixed effects and differences-in-differences with heterogeneous treatment effects: A survey. NBER Working Paper No. 29691.
Dette H., Schumann M. (2020). Difference-in-differences estimation under non-parallel trends. https://www.ruhr-uni-bochum.de/imperia/md/content/mathematik3/publications/dette_schumann2020.pdf.
Freyaldenhoven S., Hansen C., Pérez J. P., Shapiro J. M. (2021). Visualization, identification, and estimation in the linear panel event-study design. NBER Working Paper No. 29170.
Gardner J. (2021). Two-stage differences in differences. https://jrgcmu.github.io/2sdd_current.pdf.
Goodman-Bacon A. (2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225, 254–277. DOI: 10.1016/j.jeconom.2021.03.014.
Heckman J., Ichimura H., Todd P. (1998). Matching as an econometric evaluation estimator. The Review of Economic Studies, 65 (2), 261–294.
Jakiela P. (2021), Simple diagnostics for two-way fixed effects. ArXiv:2103.13229. DOI: 10.48550/arXiv.2103.13229.
Kahn-Lang A., Lang K. (2020). The promise and pitfalls of differences-in-differences: Reflections on 16 and pregnant and other applications. Journal of Business & Economic Statistics, 38 (3), 613–620. DOI: 10.1080/07350015.2018.1546591.
Keele L. J., Small D. S., Hsu J. Y., Fogarty C. B. (2019). Patterns of effects and sensitivity analysis for differences-in-differences. ArXiv:1901.01869. DOI: 10.48550/arXiv.1901.01869.
Manski C. F., Pepper J. V. (2018). How do right-to-carry laws affect crime rates? Coping with ambiguity using bounded-variation assumptions. The Review of Economics and Statistics, 100 (2), 232–244. DOI: 10.1162/REST_a_00689.
Meyer B., Viscusi W. (1995). Workers’ compensation and injury duration: Evidence from a natural experiment. The American Economic Review, 85 (3), 322–340.
Rambachan A., Roth J. (2023). A more credible approach to parallel trends. The Review of Economic Studies, 90, 2555–2591. DOI:10.1093/restud/rdad018.
Rios-Avila F. (2020). Recentered influence functions (RIFs) in Stata: RIF regression and RIF decomposition. Stata Journal, 20 (1), 51–94. DOI: 10.1177/1536867X20909690.
Roth J. (2022). Pre-test with caution: Event-study estimates after testing for parallel trends. The American Economic Review: Insights, 4 (3), 305–322. DOI: 10.1257/aeri.20210236.
Roth J., Sant’Anna P. H. (2021). Efficient estimation for staggered rollout designs. ArXiv:2102.01291. DOI: 10.48550/arXiv.2102.01291.
Roth J., Sant’Anna P. H., Bilinski A., Poe J. (2023). What’s trending in difference-in-differences? A synthesis of the recent econometrics literature. Journal of Econometrics, 235 (2), 2218–2244. DOI: 10.1016/j.jeconom.2023.03.008.
Roth J., Sant’Anna P. H. (2023). When is parallel trends sensitive to functional form? Econometrica, 91 (2), 737–747. DOI: 10.3982/ECTA19402.
Rubin D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66 (5), 670–688.
Sun L., Abraham S. (2021). Estimating dynamic treatment effects in event studies with heterogeneous treatment effects. Journal of Econometrics, 225, 175–199. DOI: 10.1016/j.jeconom.2020.09.006.
Villa J. M. (2016). diff: Simplifying the estimation of difference-in-differences treatment effects. Stata Journal, 16, 52–71.
Wald A. (1940). The fitting of straight lines if both variables are subject to error. Annals of Mathematical Statistics, 11, 284–300.
Ye T., Keele L., Hasegawa R., Small D. S. (2021). A negative correlation strategy for bracketing in difference-in-differences. ArXiv: 2006.02423. DOI: 10.48550/arXiv.2006.02423.
Wooldridge J. M. (2009). Difference-in-differences estimation. Quantile, 6, 25–47 (in Russian).
Ashenfelter O. (1978). Estimating the effect of training programs on earnings. Review of Economics and Statistics, 60 (1), 47–57.
Ashenfelter O., Card D. (1985). Using the longitudinal structure of earnings to estimate the effect of training programs. Review of Economics and Statistics, 67, 648–660.
Athey S., Imbens G. W. (2006). Identification and inference in nonlinear difference-in-differences models. Econometrica, 74 (2), 431–497. DOI: 10.1111/j.1468-0262.2006.00668.x.
Bertrand M., Duflo E., Mullainathan S. (2004). How much should we trust differences-in-differences estimates? Quarterly Journal of Economics, 119 (1), 249–275. DOI: 10.1162/003355304772839588.
Bilinski A., Hatfield L. A. (2018). Seeking evidence of absence: Reconsidering tests of model assumptions. ArXiv:1805.03273. DOI: 10.48550/arXiv.1805.03273.
Bonhomme S., Sauder U. (2011). Recovering distributions in difference-in-differences models: A comparison of selective and comprehensive schooling. Review of Economics and Statistics, 93 (2), 479–494. DOI: 10.1162/REST_a_00164.
Borusyak K., Jaravel X. (2018). Revisiting event study designs with an application to the estimation of the marginal propensity to consume. https://scholar.harvard.edu/files/borusyak/files/borusyak_jaravel_event_studies.pdf.
Borusyak K., Jaravel X., Spiess J. (2021). Revisiting event study designs: Robust and efficient estimation. ArXiv:2108.12419. DOI: 10.48550/arXiv.2108.12419.
Butts K. (2021). did2s: Two-stage difference-in-differences following Gardner (2021). R package version 1.0.2. https://cran.r-project.org/web/packages/did2s/vignettes/Two-Stage-Difference-in-Differences.html.
Callaway B., Sant’Anna P. H. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225, 200–230. DOI: 10.1016/j.jeconom.2020.12.001.
Callaway B., Li T. (2019). Quantile treatment effects in difference in differences models with panel data. Quantitative Economics, 10 (4), 1579–1618. DOI: 10.3982/QE935.
De Chaisemartin C., D’Haultfoeuille X. (2018). Fuzzy differences-in-differences. The Review of Economic Studies, 85 (2), 999–1028. DOI: 10.1093/restud/rdx049.
De Chaisemartin C., D’Haultfoeuille X. (2020). Two-way fixed effects estimators with heterogeneous treatment effects. American Economic Review, 110 (9), 2964–2996.
De Chaisemartin C., D’Haultfoeuille X. (2021). Difference-in-differences estimators of intertemporal treatment effects. NBER Working Paper No. 29873.
De Chaisemartin C., D’Haultfoeuille X. (2022). Two-way fixed effects and differences-in-differences with heterogeneous treatment effects: A survey. NBER Working Paper No. 29691.
Dette H., Schumann M. (2020). Difference-in-differences estimation under non-parallel trends. https://www.ruhr-uni-bochum.de/imperia/md/content/mathematik3/publications/dette_schumann2020.pdf.
Freyaldenhoven S., Hansen C., Pérez J. P., Shapiro J. M. (2021). Visualization, identification, and estimation in the linear panel event-study design. NBER Working Paper No. 29170.
Gardner J. (2021). Two-stage differences in differences. https://jrgcmu.github.io/2sdd_current.pdf.
Goodman-Bacon A. (2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225, 254–277. DOI: 10.1016/j.jeconom.2021.03.014.
Heckman J., Ichimura H., Todd P. (1998). Matching as an econometric evaluation estimator. The Review of Economic Studies, 65 (2), 261–294.
Jakiela P. (2021), Simple diagnostics for two-way fixed effects. ArXiv:2103.13229. DOI: 10.48550/arXiv.2103.13229.
Kahn-Lang A., Lang K. (2020). The promise and pitfalls of differences-in-differences: Reflections on 16 and pregnant and other applications. Journal of Business & Economic Statistics, 38 (3), 613–620. DOI: 10.1080/07350015.2018.1546591.
Keele L. J., Small D. S., Hsu J. Y., Fogarty C. B. (2019). Patterns of effects and sensitivity analysis for differences-in-differences. ArXiv:1901.01869. DOI: 10.48550/arXiv.1901.01869.
Manski C. F., Pepper J. V. (2018). How do right-to-carry laws affect crime rates? Coping with ambiguity using bounded-variation assumptions. The Review of Economics and Statistics, 100 (2), 232–244. DOI: 10.1162/REST_a_00689.
Meyer B., Viscusi W. (1995). Workers’ compensation and injury duration: Evidence from a natural experiment. The American Economic Review, 85 (3), 322–340.
Rambachan A., Roth J. (2023). A more credible approach to parallel trends. The Review of Economic Studies, 90, 2555–2591. DOI:10.1093/restud/rdad018.
Rios-Avila F. (2020). Recentered influence functions (RIFs) in Stata: RIF regression and RIF decomposition. Stata Journal, 20 (1), 51–94. DOI: 10.1177/1536867X20909690.
Roth J. (2022). Pre-test with caution: Event-study estimates after testing for parallel trends. The American Economic Review: Insights, 4 (3), 305–322. DOI: 10.1257/aeri.20210236.
Roth J., Sant’Anna P. H. (2021). Efficient estimation for staggered rollout designs. ArXiv:2102.01291. DOI: 10.48550/arXiv.2102.01291.
Roth J., Sant’Anna P. H., Bilinski A., Poe J. (2023). What’s trending in difference-in-differences? A synthesis of the recent econometrics literature. Journal of Econometrics, 235 (2), 2218–2244. DOI: 10.1016/j.jeconom.2023.03.008.
Roth J., Sant’Anna P. H. (2023). When is parallel trends sensitive to functional form? Econometrica, 91 (2), 737–747. DOI: 10.3982/ECTA19402.
Rubin D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66 (5), 670–688.
Sun L., Abraham S. (2021). Estimating dynamic treatment effects in event studies with heterogeneous treatment effects. Journal of Econometrics, 225, 175–199. DOI: 10.1016/j.jeconom.2020.09.006.
Villa J. M. (2016). diff: Simplifying the estimation of difference-in-differences treatment effects. Stata Journal, 16, 52–71.
Wald A. (1940). The fitting of straight lines if both variables are subject to error. Annals of Mathematical Statistics, 11, 284–300.
Ye T., Keele L., Hasegawa R., Small D. S. (2021). A negative correlation strategy for bracketing in difference-in-differences. ArXiv: 2006.02423. DOI: 10.48550/arXiv.2006.02423.
