Highlighted Publications

Dr. Heffernan is most well-known for ASSISTments, an online learning platform that provides assistance and assessment to teachers and students. The following paper provides a general overview of ASSISTments:  

• J20 Heffernan, N. & Heffernan, C. (2014). The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching. International Journal of Artificial Intelligence in Education. 24(4), 470-497. Link to the Springer version DOI 10.1007/s40593-014-0024-x. The Special Issue focused on landmark systems. 

SRI International found ASSISTments to be effective in increasing student achievement. (Dr. Heffernan is not an author) and got the study approved by the What Works Clearinghouse.  

• Roschelle, J., Feng, M., Murphy, R. & Mason, C. (2016). Online Mathematics Homework Increases Student Achievement. AERA OPEN. Vol. 2, No. 4, 1–12. DOI: 10.1177/2332858416673968. The corrected version of this paper is available here.

  • A longer version was published at the JREE with an effect size of .22 

    • Murphy, R., Roschelle, J., Feng, M. & Mason, C. (2020)  Investigating Efficacy, Moderators and Mediators for an Online Mathematics Homework Intervention. Journal of Research on Educational Effectiveness. Volume 13(2). https://doi.org/10.1080/19345747.2019.1710885- Author copy

    • The results of a second replication done in North Carolina found similar results but also that ASSISTments started closing achievement gaps (for low-income and non-white students):

      • • Feng, M., Brezack, N., Huang, C., & Collins, K. (2025). Long-term effects of an online math tool on U.S. adolescents' achievement. British Journal of Educational Technology, 56, 2404–2427. https://doi.org/10.1111/bjet.13579

    • They also have done a cost effectiveness analysis that shows ASSISTments is 70 times more cost effective than Saga Tutoring:

      • • Feng, M., Weiser, G., & Collins, K. (2024) Cost and Cost Effectiveness of ASSISTments Online Math Support Analysis From a Randomized Controlled Study in Middle School.   Technical Report posted to the WestEd website. 

Dr. Heffernan is the director of E-TRIALS (formerly the ASSISTments Testbed), a platform for conducting open science for education researchers.

• J21 Ostrow, K.S., Heffernan, N.T., & Williams, J.J. (2017). Tomorrow’s EdTech Today: Establishing a Learning Platform as a Collaborative Research Tool for Sound Science. Teachers College Record, Volume 119 Number 3, 2017, 1-36.

  • You can see the list of over a dozen published papers by external researcher here.  We published a single articles with the results of over 50 experiments on a dozen different topics see here.

    •  Prihar, E., Syed, M., Ostrow, K., Shaw, S., Sales, A., & Heffernan, N. (2022). Exploring Common Trends in Online Educational Experiments. Proceedings of the 15th International Educational Data Mining Conference. Held in Durham, England., July 2022. Winner of "Best Data Set" Award.  Dr. Heffernan is well known for the idea that we should be able to crowdsource from teachers.

Heffernan has embraced the idea of using ASSISTments as a crowd-sourcing look, taking ideas from teachers and trying to see who to give what to each student. 

• CP95 Patikorn, T. & Heffernan, N. T. (2020, August 12). Effectiveness of Crowd-Sourcing On-Demand Tutoring from Teachers in Online Learning Platforms. Proceedings of the Seventh ACM Conference on Learning @ Scale (L@S). Pages 115–124. https://doi.org/10.1145/3386527.3405912. Winner of Best Student Paper Award. 

An earlier example of crowdsourcing is this one:

• • Williams, J. J., Kim, J., Rafferty, A., Maldonado, S., Gajos, K. Z., Lasecki, W. S. & Heffernan, N. T. (2016). Axis: Generating explanations at scale with learnersourcing and machine learning. Proceedings of the Third (2016) ACM Conference on Learning @ Scale pp 379-388. (Acceptance Rate = 23%). 

ASSISTments is an accurate assessor of student knowledge and can predict state test scores by accounting for hints needed and attempts made.

• J8  Feng, M., Heffernan, N.T., & Koedinger, K.R. (2009). Addressing the assessment challenge in an Intelligent Tutoring System that tutors as it assesses. The Journal of User Modeling and User-Adapted Interaction. 19, 243-266. (Based on CP15). Publisher Copy: https://doi.org/10.1007/s11257-009-9063-7: Author Copy  Won the "James Chen" Best Journal Paper of the Year award at UMUAI 

ASSISTments has a demonstrated capacity to adapt to users.

• CP25 Razzaq, L. & Heffernan, N. (2009). To Tutor or Not to Tutor: That is the Question. In Dimitrova, Mizoguchi, du Boulay & Graesser (Eds.) Proceedings of the 2009 Artificial Intelligence in Education Conference. IOS Press, 457-464. Honorable Mention for Best Paper First Authored by a Student.

The following papers assess the impacts of treatment effects:

• SP24 Sales, A. C., Botelho, A. F., Wu, E., Gagnon-Bartsch, J., Miratrix, L., Patikorn, T. & Heffernan, N. T. (2018). Residualization Methods to Better Estimate Treatment Effects in Randomized Controlled Trials. Presented at the Conference on Digital Experimentation (CODE) held at MIT. Abstract. View recorded talk here. Here in youtube.

• CP88 Sales, A., Botelho, A. F., Patikorn, T., & Heffernan, N. T. (2018, July). Using Big Data to Sharpen Design-Based Inference in A/B Tests. In Proceedings of the Eleventh International Conference on Educational Data Mining, 479-485. Retrieved from  https://files.eric.ed.gov/fulltext/ED593197.pdf  Corrected Version is here 

Dr. Heffernan uses Bayesian Networks to Model Student Knowledge.

• CP40 Pardos, Z. & Heffernan, N. (2010). Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing. In P. De Bra, A. Kobsa, D. Chin, (Eds.) The 18th Proceedings of the International Conference on User Modeling, Adaptation and Personalization. Springer-Verlag, 255-266. Nominee for Best Student Paper. 

  • Over 100 papers have used this, and other publically released ASSSITment data sets. See details here. 

Dr. Heffernan is also well known his work detecting “gaming” behaviors.

• J6  Baker, R., Walonoski, J., Heffernan, T., Roll, I., Corbett, A. & Koedinger, K. (2008). Why students engage in "Gaming the System" behavior in interactive learning environments. Journal of Interactive Learning Research (JILR), 19(2), 185-224. (Based on CP12 and PP5). 

  • • This work led to a handful of later papers about student emotions while using ASSISTments.