Gagnon & Sales IES grant

In 2016 Dr Heffernan's lab published upon, and released, the first data set that had the data from 22 experiments. Dr Heffernan was interested in strong methods that could allow us better ways to measure the effects of these experiments.

2018 at the Atlantic Causal Inference conference Anthony and Adam met Johann Gagnon-Bartsch and Luke Miratrix who were interested in this issue and we're already doing great work in this area. We decided to try to collaborate. That collaboration has resulted in a working paper (see below) under submission and an IES grant:

  1. We have a working paper Precise Unbiased Estimation in Randomized Experiments using Auxiliary Observational Data. A. Gagnon-Bartsch*, A. C. Sales*, E. Wu, A. E. Botelho, L. W. Miratrix, N. T. Heffernan

  2. This has now resulted in this IES grant that is what IES calls a "Methodological Innovation" grant line.

Open Data

Side note: Dr Heffernan is proud of the fact that he anonymized the data and shares it with others. Many others have used these released datasets. In particular, for this data set of experiments, Rafferty, Ying & Williams (2019) JEDM paper, used this data set.


  • Sales, A.C.,Botelho, A.F., Gagnon-Bartsch, J., Heffernan, N.T., Miratrix, L., Patikorn, T., & Wu,E.Using Machine Learning and Auxiliary Data for Precise, Unbiased, Causal Inference., Contributed Presentation at: The Society for Research in Educational Effectiveness Spring Conference, Washington, DC, March 2019; The Atlantic Causal Inference Conference, Montreal, Canada, May, 2019.

  • Botelho, A., Sales, A., Patikorn, T. & Heffernan, N. (2019) The ASSISTments TestBed: Opportunities and Challenges of Experimentation in Online Learning Platforms. Workshop on Learning Anaystic Services to Support Persilzied Learning & Assessment at Scale.



  • S. Zhao and N. Heffernan. Estimating individual treatment effects from educational studies with residual counterfactual networks. In 10th International Conference on Educational Data Mining, 2017.

Early 2016