Publications

Highlighted Publications

Dr. Heffernan is most well-known for ASSISTments, an online learning platform that provides assistance and assessment to teachers and students. The following paper provides a general overview of ASSISTments:  

• J20 Heffernan, N. & Heffernan, C. (2014). The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching. International Journal of Artificial Intelligence in Education. 24(4), 470-497. Link to the Springer version DOI 10.1007/s40593-014-0024-x. The Special Issue focused on landmark systems. 

SRI International found ASSISTments to be effective in increasing student achievement. (Dr. Heffernan is not an author) and got the study approved by the What Works Clearinghouse.  

• Roschelle, J., Feng, M., Murphy, R. & Mason, C. (2016). Online Mathematics Homework Increases Student Achievement. AERA OPEN. Vol. 2, No. 4, 1–12. DOI: 10.1177/2332858416673968. The corrected version of this paper is available here.

  • A longer version was published at the JREE with an effect size of .22 

    • Murphy, R., Roschelle, J., Feng, M. & Mason, C. (2020)  Investigating Efficacy, Moderators and Mediators for an Online Mathematics Homework Intervention. Journal of Research on Educational Effectiveness. Volume 13(2). https://doi.org/10.1080/19345747.2019.1710885- Author copy

    • The results of a second replciaiton done in North Carolina found similar results but also that ASSISTments started closeing achievement gaps (for low-income and non-white students)

      • Feng, M., Huang, C., & Collins, K. (2023). Promising Long Term Effects of ASSISTments Online Math Homework Support. In International Conference on Artificial Intelligence in Education, A Late Breaking Result. pp. 212-217. Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-36336-8_32 Author PDF.

        • A more detailed report was published later :

          • Feng, M., Huang, C.-W., & Collins, K. (2023). Technology-based support shows promising longterm impact on math learning: Initial results from a randomized controlled trial in middle schools. WestEd. https://www.wested.org/resources/technology-based-support-shows-promising-long-term-impact-on-math-learning-2/ 

Dr. Heffernan is the director of E-TRIALS (formerly the ASSISTments Testbed), a platform for conducting open science for education researchers.

• J21 Ostrow, K.S., Heffernan, N.T., & Williams, J.J. (2017). Tomorrow’s EdTech Today: Establishing a Learning Platform as a Collaborative Research Tool for Sound Science. Teachers College Record, Volume 119 Number 3, 2017, 1-36.

  • You can see the list of over a dozen published papers by external researcher here.  We published a single articles with the results of over 50 experiments on a dozen different topics see here.

    •  Prihar, E., Syed, M., Ostrow, K., Shaw, S., Sales, A., & Heffernan, N. (2022). Exploring Common Trends in Online Educational Experiments. Proceedings of the 15th International Educational Data Mining Conference. Held in Durham, England., July 2022. Winner of "Best Data Set" Award.  Dr. Heffernan is well known for the idea that we should be able to crowdsource from teachers.

Heffernan has embraced the idea of using ASSISTments as a crowd-sourcing look, taking ideas from teachers and trying to see who to give what to each student. 

• CP95 Patikorn, T. & Heffernan, N. T. (2020, August 12). Effectiveness of Crowd-Sourcing On-Demand Tutoring from Teachers in Online Learning Platforms. Proceedings of the Seventh ACM Conference on Learning @ Scale (L@S). Pages 115–124. https://doi.org/10.1145/3386527.3405912. Winner of Best Student Paper Award. 

An earlier example of crowdsourcing is this one:

• • Williams, J. J., Kim, J., Rafferty, A., Maldonado, S., Gajos, K. Z., Lasecki, W. S. & Heffernan, N. T. (2016). Axis: Generating explanations at scale with learnersourcing and machine learning. Proceedings of the Third (2016) ACM Conference on Learning @ Scale pp 379-388. (Acceptance Rate = 23%). 

ASSISTments is an accurate assessor of student knowledge and can predict state test scores by accounting for hints needed and attempts made.

• J8  Feng, M., Heffernan, N.T., & Koedinger, K.R. (2009). Addressing the assessment challenge in an Intelligent Tutoring System that tutors as it assesses. The Journal of User Modeling and User-Adapted Interaction. 19, 243-266. (Based on CP15). Publisher Copy: https://doi.org/10.1007/s11257-009-9063-7: Author Copy  Won the "James Chen" Best Journal Paper of the Year award at UMUAI 

ASSISTments has a demonstrated capacity to adapt to users.

• CP25 Razzaq, L. & Heffernan, N. (2009). To Tutor or Not to Tutor: That is the Question. In Dimitrova, Mizoguchi, du Boulay & Graesser (Eds.) Proceedings of the 2009 Artificial Intelligence in Education Conference. IOS Press, 457-464. Honorable Mention for Best Paper First Authored by a Student.

The following papers assess the impacts of treatment effects:

• SP24 Sales, A. C., Botelho, A. F., Wu, E., Gagnon-Bartsch, J., Miratrix, L., Patikorn, T. & Heffernan, N. T. (2018). Residualization Methods to Better Estimate Treatment Effects in Randomized Controlled Trials. Presented at the Conference on Digital Experimentation (CODE) held at MIT. Abstract. View recorded talk here. Here in youtube.

• CP88 Sales, A., Botelho, A. F., Patikorn, T., & Heffernan, N. T. (2018, July). Using Big Data to Sharpen Design-Based Inference in A/B Tests. In Proceedings of the Eleventh International Conference on Educational Data Mining, 479-485. Retrieved from  https://files.eric.ed.gov/fulltext/ED593197.pdf  Corrected Version is here 

Dr. Heffernan uses Bayesian Networks to Model Student Knowledge.

• CP40 Pardos, Z. & Heffernan, N. (2010). Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing. In P. De Bra, A. Kobsa, D. Chin, (Eds.) The 18th Proceedings of the International Conference on User Modeling, Adaptation and Personalization. Springer-Verlag, 255-266. Nominee for Best Student Paper. 

  • Over 100 papers have used this, and other publically released ASSSITment data sets. See details here. 

Dr. Heffernan is also well known his work detecting “gaming” behaviors.

• J6  Baker, R., Walonoski, J., Heffernan, T., Roll, I., Corbett, A. & Koedinger, K. (2008). Why students engage in "Gaming the System" behavior in interactive learning environments. Journal of Interactive Learning Research (JILR), 19(2), 185-224. (Based on CP12 and PP5). 

  • • This work led to a handful of later papers about student emotions while using ASSISTments. 

Journal Articles.  

J37 Cheng, L., Croteau, E., Baral, S., Heffernan, C., & Heffernan, N. (2024). Facilitating Student Learning With a Chatbot in an Online Math Learning Platform. Journal of Educational Computing Research, 0(0). Author's Copy https://doi.org/10.1177/07356331241226592 

J36 Li, H., Wanli, C., Baral, S., & Heffernan, N. T. (submitted). Exploring the Multi-Modality Bias Arising from Privacy Leakage in an Automatic Feedback System Enhanced by Generative Artificial Intelligence. British Journal of Educational Technology.

J35 Lu, X., Wang, W., Motz, B. A., Ye, W., & Heffernan, N. T. (2023). Immediate text-based feedback timing on foreign language online assignments: How immediate should immediate feedback be? Computers and Education Open, 5. https://doi.org/10.1016/j.caeo.2023.100148. Author copy

J35 Sales, A.C., Gagnon-Bartsch, J.A., Prihar, E.B., & Heffernan, N.T. (2023). Using Auxiliary Data to Boost Precision in the Analysis of A/B Tests on an Online Educational Platform: New Data and New Results. Journal of Educational Data Mining. Accepted. https://arxiv.org/abs/2306.06273 

J34 Gagnon-Bartsch, J. A., Sales, A.C.,*, Wu, E.,  Botelho,  A. F.,  Erickson,  J. A., Miratrix,  L. W., & Heffernan, N. T. (Accepted 2023). Precise unbiased estimation in randomized experiments using auxiliary observational data. Journal of Casual Inference. https://arxiv.org/abs/2105.03529 

J33 Huang, W., Labille, K., Wu, X., Lee, D. & Heffernan, N. (2022). Achieving User-Side Fairness in Contextual Bandits. Human-Centric Intelligent Systems. Springer.  https://link.springer.com/article/10.1007/s44230-022-00008-w 

J32 Lu, X., Sales, A., & Heffernan, N. T. (2021). Immediate Versus Delayed Feedback on Learning: Do People’s Instincts Really Conflict With Reality?. Journal of Higher Education Theory and Practice, 21(16). https://doi.org/10.33423/jhetp.v21i16.4925 

J31 Lu, X., Wang, W., Motz, B., Ye, W., & Heffernan, N. T. (submitted for publication). How Immediate Should Immediate Feedback Be? Immediate Feedback Timing and its Effects on Learning Performance of Written Homework Assignments. Computers & Education Open.

J30 Botelho, A., Baral, S., Erickson, J., Benachamardi, P., & Heffernan, N. (2023). Leveraging natural language processing to support automated assessment and feedback for student open responses in mathematics. Journal of Computer Assisted Learning. https://doi.org/10.1111/jcal.12793     

J29 McCarthy, K. S., Crossley, S. A., Meyers, K., Boser, U., Allen, L. K., Chaudhri, V. K., Collins-Thompson, K., D’Mello, S., De Choudhury, M., Garg, K., Goel, A., Gosha, K., Heffernan, N., Hooper, M. A., Hyman, E., Jarratt, D. C., Khalil, D., Kizilcec, R. F., Litman, D., Malatinszky, A., Marks, K., McNamara, D. S., Menko, R., Palermo, C., Porcaro, D., Roscoe, R., Shapiro, S., Khanh-Phoung, T., Trumbore, A. M., White, C., Wong, W., Yang, D., & Zampieri, M. (in press). Toward more effective and equitable learning: Identifying barriers and solutions for the future of online education. Technology, Mind, & Behavior.  

J28 Patikorn, T., Baker, R. S., & Heffernan, N. T. (2020). ASSISTments Longitudinal Data Mining Competition Special Issue: A Preface. JEDM | Journal of Educational Data Mining, 12(2), i-xi. https://doi.org/10.5281/zenodo.4008048  This is part of a Special Issue that Heffernan et al organized . The competition web site where 70+ researchers participated is here. 

J27 Lu, X., Ostrow, K., & Heffernan, N. (2020). Save Your Strokes: Chinese Handwriting Practice Makes for Ineffective Use of Instructional Time in Second Language. AERA Open.  Volume 6(2)  https://doi.org/10.1177/2332858419890326

J26  Graesser, A. C., Hu, X., Nye, B. D., VanLehn, K.,Kumar, R., Heffernan, C., Heffernan, C., et al. (2018). ElectronixTutor: an intelligent tutoring system with multiple learning resources for electronics. International Journal of STEM Education, 5: 15. 

J25  Kai, S., Almeda, M. V., Baker, R., Heffernan, C., & Heffernan, N. (2018). Decision Tree Modeling of Wheel- Spinning and Productive Persistence in Skill Builders. JEDM | Journal of Educational Data Mining, 10(1), 36-71.  Errata and New version 

J24 McGuire, P., Tu, S., Logue., M., Mason, C., Ostrow, K. (2017). Counterintuitive effects of online feedback in middle school math: results from a randomized controlled trial in ASSISTments. Educational Media International. Taylor Francis. Vol Pages 1-14. (Heffernan was not an author, but he helped these scientists use his platform.)

J23 Inventado, P., Scupelli, P., Ostrow, K., Heffernan, N., Almeda, V., & Slater, (2018). Contextual Factors Affecting Hint Utility, International Journal of STEM Education, 5(1), 13. 

J21 Ostrow, K.S., Heffernan, N.T., & Williams, J.J. (2017). Tomorrow’s EdTech Today: Establishing a Learning Platform as a Collaborative Research Tool for Sound Science. Teachers College Record Volume 119 Number 3, 2017, p. 1-36. ID Number: 21779

J20 Heffernan, N.T., Ostrow, K.S., Kelly, K., Selent, D., Van Inwegen, E.G., Xiong, X., & Williams, J.J. (2016). The Future of Adaptive Learning: Does the Crowd Hold the Key? International Journal of Artificial Intelligence in Education. Springer New York. DOI: 10.1007/s40593-016-0094-z 

J19  Heffernan, N. & Heffernan, C. (2014). The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching. International Journal of Artificial Intelligence in Education. 24 (4), 470-497. DOI 10.1007/s40593-014-0024-x. The Special Issue focused on landmark systems.

J18  Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., Heffernan, C. (2014). Population validity for Educational Data Mining models: A case study in affect detection. British Journal of Educational Technology, 45(3), 487-501. DOI: 10.1111/bjet.12156

J17  Pardos, Z.A., Gowda, S. M., Baker, R. S.J.D., & Heffernan, N. T. (2012). The Sum is Greater than the Parts: Ensembling Models of Student Knowledge in Educational Software. ACM’s Knowledge Discovery and Datamining Explorations, 13(2), 37-44.

J16  Gong, Y, Beck, J. E., Heffernan, N. T. (2011). How to Construct More Accurate Student Models: Comparing and Optimizing Knowledge Tracing and Performance Factor Analysis. International Journal of Artificial Intelligence in Education. 21, 27-46.

J15  Pardos, Z., Dailey, M. & Heffernan, N. (2011). Learning what works in ITS from non-traditional randomized controlled trial data. The International Journal of Artificial Intelligence in Education. 21, 47-63.

J14  Koedinger, K., McLaughlin, E. & Heffernan, N. (2010). A Quasi-Experimental Evaluation of an On-line Formative Assessment and Tutoring System. Journal of Educational Computing Research. Baywood Publishing. 4, 489-510 

J13  Baker, R., Goldstein, A., Heffernan, N. (2011). Detecting the Moment of Learning. International Journal of Artificial Intelligence in Education, 21(1-2), 5-25. (Get the data here)

J12  Broderick, Z., O’Connor, C., Mulcahy, C., Heffernan, N. & Heffernan, C. (2011). Increasing Parent Engagement in Student Learning Using an Intelligent Tutoring System.  Journal of Interactive Learning Research, 22(4), 523-550. Chesapeake, VA: AACE. 

J11 Militello, M., & Heffernan, N. (2009). Which one is "just right"? What educators should know about formative assessment systems. International Journal of Educational Leadership Preparation, 4(3), 1-8.

J10  Feng, M., Heffernan, N.T., Heffernan, & C., Mani, M. (2009). Using Mixed-Effects Modeling to Analyze Different Grain-Sized Skill Models. IEEE Transactions on Learning Technologies, 2(2), 79-92. Featured Best Article of 8 articles in the edition (Based on PP8 and WP15)

J9 Razzaq, L., Patvarczki, J., Almeida, S.F., Vartak, M., Feng, M., Heffernan, N.T. and Koedinger, K. (2009). The ASSISTment builder: Supporting the Life-cycle of ITS Content Creation. IEEE Transactions on Learning Technologies Special Issue on Real-World Applications of Intelligent Tutoring Systems. 2(2) 157-166 (Based on YRP5)

J8 Feng, M., Heffernan, N.T., & Koedinger, K.R. (2009). Addressing the assessment challenge in an Intelligent Tutoring System that tutors as it assesses. The Journal of User Modeling and User-Adapted Interaction. 19, 243-266. (Based on CP15) Best Paper of the Year (See Award #20 above). Mentioned in National Ed. Tech Plan (See Award #19).

J7  Mendicino, M., Razzaq, L. & Heffernan, N. T. (2009). Improving Learning from Homework Using Intelligent Tutoring Systems. Journal of Research on Technology in Education (JRTE). 41(3), 331-346.

J6  Baker, R., Walonoski, J., Heffernan, T., Roll, I., Corbett, A. & Koedinger, K. (2008). Why students engage in "Gaming the System" behavior in interactive learning environments Journal of Interactive Learning Research (JILR).19(2), 185-224 (Based on CP12 and PP5)

J5  Razzaq, L., Heffernan, N., Feng, M., & Pardos Z. (2007). Developing Fine-Grained Transfer Models in the ASSISTment System. Journal of Technology, Instruction, Cognition, and Learning. 5(3), 289-304.

J4  Feng, M., & Heffernan, N.T. (2007). Towards live informing and automatic analyzing of student learning: Reporting in the Assistment system. Journal of Interactive Learning Research (JILR) 18(2), 207-230. (Based on W12)

J3  Feng, M., & Heffernan, N.T. (2006). Informing teachers live about student learning: Reporting in the Assistment system Technology, Instruction, Cognition, and Learning Journal. 3(1-2), 63. (Based on W12)

J2  Heffernan, N. T., Koedinger, K. & Razzaq, L. (2008). Expanding the model-tracing architecture: A 3rd generation intelligent tutor for Algebra symbolization. The International Journal of Artificial Intelligence in Education. 18(2), 153-178 (Builds upon CP8 and CP1-4).

J1  Heffernan, N. T., & Koedinger, K. R. (2002). Results from a Web-Based Tutor for Writing Algebra Expressions for Word-Problems Sciences et Techniques Educatives. 9(1-2), 11-35. (This French language journal has translated our paper into French. My translation of this Journal’s title is "Educational Sciences and Technology.") http://www.inrp.fr/atief/ste/ste-res-vol9.htm (Based on D1)

Book Chapters

BC12 Cheng, L., Prihar, E., Baral, S., Gurung, A., Botelho, A. T., Haim A., Heffernan, C., Patikorn, T., Sales, A., & Heffernan, N. T. (2023). Authoring Tools for Crowdsourcing from Teachers to Enhance Intelligent Tutoring Systems. In Sinatra, A.M., Graesser, A.C., Hu, X., Townsend, L.N. and Rus, V. (Eds.), Design Recommendations for Intelligent Tutoring Systems: Volume 11 - Professional Career Education, Orlando, FL: US Army Combat Capabilities Development Command - Soldier Center. ISBN 978-0-9977258-5-8. Available at: https://gifttutoring.org/documents/167 

BC11  Rodrigo, M.M.T., Vassileva, J.,...Heffernan, N., & et al. (2023). The Great Challenges and Opportunities of the Next 20 Years. In Benedict du Boulay , Antonija Mitrovic, and Kalina Yacef's (eds.). The Handbook of Artificial Intelligence in Education.   

BC10 Prihar, E. & Heffernan, N. (2023). Crowdsourcing Paves the Way for Personalized Learning. In B. du Boulay, A. Mitrovic, K. Yacef, (Eds.), Handbook of Artificial Intelligence in Education. (pp. 632-635). Edward Elgar Publishing. 

BC9  Lu, X., Ostrow, K. S., Yang, Q., & Heffernan, N. T. (2022 accepted for publication; passed review). Save Your Strokes: Further Studies on the efficiency Efficiency of Learning Chinese Words Without Practicing Handwriting Practices. In Chu, C., Coss, M., & Zhang, P. N. (Eds.),  Transforming L2 Hanzi Teaching & Learning in the Age of Digital Writing: Theory and Pedagogy ( Chinese name of the book is《电写时代汉字教学的理论与实践》). Routledge, UK.

BC8  Kelly, K. & Heffernan, N. (2021). Technology enhanced formative assessment increased the efficacy of the homework review.  In Jiao & Lissitz (Eds), Enhancing Effective Instruction and Learning Using Assessment Data.  MARC  ISBN 978-1-64802-626-3.

BC7  Ostrow, K.S. & Heffernan, N.T. (2019). Advancing the State of Online Learning: Stay Integrated, Stay Accessible, Stay Curious. In Robert Feldman (Ed.) Learning Science: Theory, Research, and Practice. McGraw Hill. p. 201-227. ISBN 1260458008. PDF 

BC6 Botelho, A. & Heffernan, N., (2019). Crowdsourcing feedback to support teachers and students. In Sinatra, A.M., Graesser, A.C., Hu, X., Brawner, K., and Rus, V. (Eds.). (2019). Design Recommendations for Intelligent Tutoring Systems: Volume 7 - Self-Improving Systems. Orlando, FL: U.S. Army Research Laboratory. ISBN 978-0-9977257-7-3. Available at: https://gifttutoring.org/documents/  Page 101-108.

BC5 Heffernan, N., Militello, M, Heffernan, C., & Decoteau, M. (2012). Effective and meaningful use of educational technology: three cases from the classroom. In C. Dede & J. Richards (Eds.). Digital Teaching Platforms, 88-102. Columbia, NY: Teachers College Press.

BC4 Razzaq, L. & Heffernan, N. (2010). Open content Authoring Tools. In Nkambou, Bourdeau & Mizoguchi (Eds.) Advanced in Intelligent Tutoring Systems.pp 425-439. Berlin: Springer Verlag.

BC3 Pardos, Z. A., Heffernan, N. T., Anderson, B., Heffernan, L. C. (2010). Using Fine-Grained Skill Models to Fit Student Performance with Bayesian Networks. Chapter in C. Romero, S. Ventura, S. R. Viola, M. Pechenizkiy and R. S. J. Baker. Handbook of Educational Data Mining. Boca Raton, Florida: Chapman & Hall/CRC Press.

BC2 Feng, M., Heffernan, N.T., & Koedinger, K.R. (2010). Student Modeling in an Intelligent Tutoring System. In Stankov, Glavinc, and Rosic. (Eds.) Intelligent Tutoring Systems in E-learning Environments: Design, Implementation and Evaluation, 208-236. Hershey, PA: Information Science Reference. (Based on W10, W11 and W12).

BC1 Razzaq, Feng, Heffernan, Koedinger, Nuzzo-Jones, Junker, Macasek, Rasmussen, Turner & Walonoski. (2007). A Web-based authoring tool for intelligent tutors: Assessment and instructional assistance. In Nadia Nedjah, Luiza deMacedo Mourelle, Mario Neto Borges and Nival Nunesde Almeida (Eds). Intelligent Educational Machines. Intelligent Systems Engineering Book Series, 23-49. Berlin: Springer Verlag. 

Short Papers

In computer science, there are many times where a full paper gets downgraded to a "short" paper. For instance, at the Educational Data Mining Conference, a full paper is typically 8-10 pages and has an acceptance rate of 30% (those papers are listed above). These "Short Papers" listed below typically have an acceptance rate of 50% so they should not be considered in the same league as the work listed above (e.g., In 2011, the acceptance rates for short papers was 46%, which is considerably higher than the 33% acceptance rate for “Full Papers”).

SP33 Lee, M., Siedahmed, A., & Heffernan, N. (2024). Expert Features for a Student Support Recommendation Contextual Bandit Algorithm. In Proceedings of the 14th Learning Analytics and Knowledge Conference (LAK '24). Association for Computing Machinery, New York, NY, USA, 864–870. https://doi.org/10.1145/3636555.3636909 

SP32 Zhang, M., Heffernan, N., Lan, A. (2023) Modeling and Analyzing Scorer Preferences in Short-Answer Math Questions. Educational Data Mining. https://files.eric.ed.gov/fulltext/ED630868.pdf 

SP31  Baral, S., Santhanam, A., Botelho, A.F., Santhanam, A., Gurung, A., Cheng, L., & Heffernan, N. (2023). Auto-scoring Student Responses with Images in Mathematics. In The Proceedings of the 16th International Conference on Educational Data Mining. Submitted paper. Final paper.  

SP30 Prihar, E., Vanacore, K., Sales, A., & Heffernan, N. (2023). Effective Evaluation of Online Learning Interventions with Surrogate Measures. In The Proceedings of the 16th International Conference on Educational Data Mining. Submitted paper. Final paper.

SP29 Lee, M.P., Croteau, E., Gurung, A., Botelho, A.F., & Heffernan, N. (2023). Knowledge Tracing Over Time: A Longitudinal Analysis. In The Proceedings of the 16th International Conference on Educational Data Mining. Submitted paper. Final paper. 

SP28 Baral, S., Seetharaman, K., Botelho, A.,  Wang, A., Heineman, G.,& Heffernan, N. (2022)  Enhancing auto-scoring of student open-responses in the presence of mathematical terms and expressions. AIED2022. Submitted longer version - Shorter Final version  Talk

SP27 Prihar,E. & Heffernan, N. (2021). A Novel Algorithm for Aggregating Crowdsource Opinions.  In Hsiao,  Sahebi, Bouchet & Vie (eds). Proceedings of the 14th International Conference on Educational Data Mining (EDM2021).  Pages 547-552. https://educationaldatamining.org/EDM2021/virtual/static/pdf/EDM21_paper_63.pdf 

SP26 Razzaq, R., Ostrow, K. &  Heffernan, N. (2020) Effect of Immediate Feedback on Math Achievement in Secondary Education AIED2020.  In Bittencourt et al, The 21st Proceedings of the International Conference on Artificial Intelligence in Education (AIED). pp. 263-267. doi: https://doi.org/10.1007/978-3-030-52240-7_48  Longer Blinded Version 

SP25 Patikorn, T., Deisadze, D., Grande, L., Yu, Z., & Heffernan, N. (2019). Generalizability of Methods for Imputing Mathematical Skills Needed to Solve Problems from Texts. In International Conference on Artificial Intelligence in Education (pp. 396-405). Springer, Cham. 

SP24 Sales, A. C., Botelho, A. F., Wu, E., Gagnon-Bartsch, J., Miratrix, L., Patikorn, T. & Heffernan, N. T. (2018). Residualization Methods to Better Estimate Treatment Effects in Randomized Controlled Trials. Presented at the Conference on Digital Experimentation (CODE) held at MIT. Abstract. View recorded talk here. Here in youtube.

SP23 Sales, A., Patikorn, T. & Heffernan, N. T. (2018) Bayesian Partial Pooling to Improve Inference Across A/B Tests in EDM. Published in the Proceeding of the Educational Data Mining Conference (EDM2018) Pp 521-524. Retrieved from http://educationaldatamining.org/files/conferences/EDM2018/EDM2018_Preface_TOC_Proceedings.pdf 

SP22  Patikorn, T, Selent, D., Beck, J., Heffernan, N., & Zhou, J. (2017). Using a Single Model Trained Across Multiple Experiments to Improve the Detection of Treatment Effects. Conference of Educational Data Mining 2017. Pp 202-207.  

SP21 Zhao, S. & Heffernan, N. (2017) Estimating Individual Treatment Effects from Educational Studies with Residual Counterfactual Networks. In the 10th International Conference on Educational Data Mining (EDM 2017).  

SP20 Zhang, L., Xiong, X., Zhao, S., Botelho, A. & Heffernan, N. (2017) Incorporating Rich Features into Deep Knowledge Tracing. In the Proceedings of the Forth (2017) ACM Conference on Learning @ Scale. Cambridge, MA. (L@S2017) ,169-172. (Acceptance Rate = 44%) PDF (A longer version is available here)

SP19 Yin, B., Patikorn, T., Botelho, A., Heffernan, N. (2017) Observing Personalizations in Learning: Identifying Heterogeneous Treatment Effects Using Causal Trees. In the Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale.(L@S2017) , 299-302. Cambridge, MA. (Acceptance Rate = 44%) 

SP18 Zhao, S., Zhang, Y., Xiong, X., Botelho, A. F., & Heffernan, N. T. (2017) A Memory-Augmented Neural Model for Automated Grading. In the Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale.(L@S2017)   Cambridge, MA. Pages 189-172 (Acceptance Rate = 44%) 

SP17 Lu, X., Xiong, X. & Heffernan, N. (2017) Experimenting Choices of Video and Text Feedbacks in Authentic Foreign Language Assignments at Scale. In the Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale.(L@S2017) , 335-338. (Acceptance Rate 44%)

SP16 Inventado, P. S., VanInwegen, E., Ostrow, K., Scupelli, P., Heffernan, N., Baker, R., Slater, S. & Ocumpaugh, J. (2016) Design Subtleties Driving Differential Attrition. The 6th International Learning Analytics & Knowledge Conference pp 284-289. 

SP15 Ostrow, K.S. & Heffernan, N. T. (2016) Studying Learning at Scale with the ASSISTments TestBed Proceedings of the Third (2016) ACM Conference on Learning @Scale, 333-334.

SP14  Inventado, P., Scupelli, P., Van Inwegen, E., Ostrow, K., Heffernan, N., Ocumpaugh, J., Baker, R., Slater, S. & Almeda, M. (2016) Hint availability slows completion times in summer work. In: Barnes, T, Chi, M, Feng, M (Eds.) In Proceedings of the 9th International Conference on Educational Data Mining, (pp. 388–393). (Acceptance Rate = 50%)

SP13 Botelho, A., Adjei, S. & Heffernan, N. (2016) Modeling Interactions Across Skills: A Method to Construct and Compare Models Predicting the Existence of Skill Relationships. In Barnes, Chi & Feng (eds) The 9th International Conference on Educational Data Mining, 292-297.

SP12 Wang, Y., Ostrow, K., Beck, J. & Heffernan, N. T. (2016) Enhancing the efficiency and reliability of group differentiation through partial credit. In the Proceedings of the Sixth International Conference on Learning Analytics & Knowledge LAK2016 , 454-458 (Acceptance Rate = Not released but maybe 50%). Materials from the study.   

SP11 Adjei, S., Boethello, A. & Heffernan, N. (2016) Predicting student performance on post-requisite skills using prerequisite skill data: an alternative method for refining prerequisite skill structures In the Proceedings of the Sixth International Conference on Learning Analytics & Knowledge LAK2016 , 469-473 (Acceptance Rate = Not released but maybe 50%). 

SP10 Ostrow, K, Donnelly, C. & Heffernan, N. (2015) Optimizing Partial Credit Algorithms to Predict Student Performance. In the Proceedings of the 8th International Conference on Educational Data Mining EDM2015, Madrid, Spain. ISBN: 978-84-606-9425-0, 404-407. (Acceptance Rate = 50%)

SP9 Castro, F., Adjei, S., Colombo, T., & Heffernan, N.T. (2015) Building Models to Predict Hint-or-Attempt Actions of  Students. In the Proceedings of the 8th International Conference on Educational Data Mining EDM2015, Madrid, Spain. ISBN: 978-84-606-9425-0, 476-479. (Acceptance Rate = 50%)

SP8 Wang, Y., Heffernan, N, & Heffernan, C. (2015). Towards better affect detectors: effect of missing skills, class features and common wrong answers. Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, 31-35.   

SP7 Van Inwegen, E., Adjei, S., Wang, Y., & Heffernan, N. (2015) An analysis of the impact of action order on future performance: the fine-grain action model. Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, 320-324. 

SP6 San Pedro, M.O., Baker, R., Heffernan, N., Ocumpaugh, J. (2015) Exploring College Major Choice and Middle School Student Behavior, Affect and Learning: What Happens to Students Who Game the System? Proceedings of the 5th International Learning Analytics and Knowledge Conference, 36-40. 

SP5  Adjei, S., Selent, D., Heffernan, N., Pardos, Z., Broaddus, A., Kingston, N. (2014). Refining Learning Maps with Data Fitting Techniques: Searching for Better Fitting Learning Maps. In John Stamper et al. (Eds) Proceedings of the 7th International Conference on Educational Data Mining, 413-414. 

SP4  Selent, D. & Heffernan, N. (2014). Reducing Student Hint Use by Creating Buggy Messages from machine Learned Incorrect Processes. In Stefan Trausan-Matu, et al. (Eds) International Conference on Intelligent Tutoring 2014.  LNCS 8474. (Acceptance Rate = 66%)

SP3  Gu, J., Wang, Y. & Heffernan, N. (2014). Personalizing Knowledge Tracing: Should We Individualize Slip, Guess, Prior or Learn rate? In Stefan Trausan-Matu, et al. (Eds) International Conference on Intelligent Tutoring 2014.  LNCS 8474. (Acceptance Rate = 66%)

SP2  Kelly, K.,  Heffernan, N., Heffernan, C.,  Goldman, S., Pellegrino, G. & Soffer, D. (2013). Estimating the Effect of Web-Based Homework. In Lane, Yacef, Mostow & Pavlik (Eds) The Artificial Intelligence in Education Conference, 824-827. (Watch the videos and related archived material  here: http://web.cs.wpi.edu/~nth/PublicScienceArchive/Kelly.htm and permanently archived at http://www.webcitation.org/6E6lv54G8) 

SP1  Feng, M., Heffernan, N., Pardos, Z. & Heffernan, C.(2011). Establishing the value of dynamic assessment in an online tutoring system. In Pechenizkiy, M., Calders, T., Conati, C., Ventura, S., Romero , C., and Stamper, J. (Eds.) Proceedings of the 4th International Conference on Educational Data Mining, 295-300.

Published 3-4 Page Papers

(aka “Poster”) in Prestigious Conferences (Acceptance rates of  50-60%).

PP49 Prihar, E., Lee, M., Hopman, M., Kalai, A., Vempala, S., Wang, A., Wickline, G., & Heffernan, N. (2023). Comparing Different Approaches to Generating Mathematics Explanations Using Large Language Models.  In: Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., Santos, O.C. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2023. Communications in Computer and Information Science, vol 1831. Springer, Cham. https://doi.org/10.1007/978-3-031-36336-8_45  Submitted PDF Final PDF

PP48 Baral, S., Botelho, A.F., Santhanam, A., Gurung, A., Erickson, J., & Heffernan, N. (2023). Investigating Patterns of Tone and Sentiment in Teacher Written Feedback Messages. Accepted to AIED 2023. Submitted PDF Final PDF

PP47 Sales, A., Prihar, E., Gagnon-Bartsch, J., Gurung, A., & Heffernan, N. (2022). More Powerful A/B Testing Using Auxiliary Data and Deep Learning.  AIED2022. Submitted Version (blind). 

PP46 Haim, A. & Heffernan, N. (2022). Student Perception on the Effectiveness of On-Demand Assistance in Online Learning Platforms. EDM Accepted Version. 

PP45 Rivera-Bergollo, R., Baral, S., Botelho, A., & Heffernan, N. (2022). Leveraging Auxiliary Data from Similar Problems to Improve Automatic Open Response Scoring. Accepted Version. EDM PDF

PP44 Erickson, J.A., Botelho, A., Peng, Z., Huang, R., Kasal, M.V., Heffernan, N. (2021) Is it Fair? Automated Open Response Grading. In Hsiao,  Sahebi, Bouchet & Vie (eds). Proceedings of the 14th International Conference on Educational Data Mining (EDM21).  Page 682-687. 

PP43 Botelho, A., Baker, R. & Heffernan, N. (2019) Machine-Learned or Expert-Engineered Features? Exploring Feature Engineering Methods in Detectors of Disengaged Behavior and Affect. In Desmarais, Lynch, Merceron & Nkambou (Eds) Proceedings of the 12th International Conference on Educational Data Mining(EDM2019)  ISBN: 978-1-7336736-0-0. pp. 508-511

PP42 Varatharaj, A., Botelho, A., Lu, W. & Heffernan, N. (2019) Hao Fa Yin: Developing Automated Audio Assessment Tools for a Chinese Language Course. In Desmarais, Lynch, Merceron & Nkambou (Eds) Proceedings of the 12th International Conference on Educational Data Mining(EDM2019)  ISBN: 978-1-7336736-0-0. pp. 663-667

PP41 Hulse, T., Harrison, A., Ostrow, K. S., Botelho, A. F., & Heffernan, N. T. (2018). Starters and Finishers: Predicting Next Assignment Completion From Student Behavior During Math Problem Solving. In Proceedings of the Eleventh International Conference on Educational Data Mining, 525-528.

PP40 Yin, B., Botelho, A. F., Patikorn, T., Heffernan, N. T., & Zou, J. (2017, June). Causal Forest vs. Naïve Causal Forest in Detecting Personalization: An Empirical Study in ASSISTments. In Proceedings of the Tenth International Conference on Educational Data Mining, 388-389. ACM

PP39 Patikorn, T., Heffernan, N., & Zhou, J. (2017). An Offline Evaluation Method for Individual Treatment Rules and How to Find Heterogeneous Treatment Effects. Conference of Educational Data Mining 2017. 

PP38 Patikorn, T., Selent, D., Heffernan, N. T., Yin, B., Botelho, A. (2016) ASSISTments Dataset for a Data Mining Competition to Improve Personalized Learning. Poster at MIT Conference on Digital Experimentation. CODE 2016.

PP37 Williams, J. J., Botelho, A., Sales, A., Heffernan, N. & Lang, C. (2016) Discovering 'Tough Love' Interventions Despite Dropout In Barnes, Chi & Feng (eds) The 9th International Conference on Educational Data Mining, 650-651.

PP36 Kelly, K. & Heffernan, N. (2016) Optimizing the Amount of Practice in an On-Line Platform. A “Work in progress” category presented at Learning at Scale 2016. Pp. 145-148. (Acceptance Rate = 59%) 

PP35  Selent, D., Patikorn, T. & Heffernan, N. T. (2016) ASSISTments Dataset from Multiple Randomized Controlled Experiments. A “Work in progress” category presented at Learning at Scale 2016. At ACM Digital Library, 181-184. (Acceptance Rate = 59%)

PP34 Wang, Y., Ostrow, K., Adjei, S. & Heffernan, N. (2016) The Opportunity Count Model: A Flexible Approach to Modeling Student Performance. A “Work in progress” category presented at Learning at Scale 2016. At ACM Digital Library. (Acceptance Rate = 59%)

PP33  Williams, J. W. & Heffernan, N. T. (2015) A Methodology for Discovering how to Adaptively Personalize to Users using Experimental Comparisons. Submitted to UMAP Late Breaking Results. Not final draft. (Acceptance Rate = 50%)

PP32  Selent, D. & Heffernan, N. T. (2015) When More Intelligent Tutoring in the Form of Buggy Messages Does Not Help  In Conati, Heffernan, Mitrovic & Verdejo (Eds) The 17th Proceedings of the Conference on Artificial Intelligence in Education, Madrid, Spain. Springer, 768-771.

PP31 Jiang, Y., Baker, R., Paquette, L., San Pedro, M. & Heffernan, N. T. (2015) Learning, Moment-by-Moment and Over the Long Term. In Conati, Heffernan, Mitrovic & Verdejo (Eds) The 17th Proceedings of the Conference on Artificial Intelligence in Education, Madrid, Spain. Springer, 654-657.

PP30 Ostrow, K. & Heffernan, N. T. (2015) The Role of Student Choice Within Adaptive Tutoring. In Conati, Heffernan, Mitrovic & Verdejo (Eds) The 17th Proceedings of the Conference on Artificial Intelligence in Education, Madrid, Spain. Springer, 752-755.

PP29 Adjei, S. A., & Heffernan, N. (2015)  Improving Learning Maps Using an Adaptive Testing System: PLACEMENTS. In Conati, Heffernan, Mitrovic & Verdejo (Eds) The 17th Proceedings of the Conference on Artificial Intelligence in Education, Madrid, Spain. Springer, 517-520.(Longer Version)

PP28  Van Inwegen, E., Wang, Y., Adjei, S. & Heffernan, N.T. (2015) The Effect of the Distribution of Predictions of User Models. In the Proceedings of the 8th International Conference on Educational Data Mining EDM2015, Madrid, Spain. ISBN: 978-84-606-9425-0. pp 620-621.

PP27  Botelho, A., Adjei, S., Wan, H. & Heffernan, N.T. (2015) Predicting Student Aptitude Using Performance History. In the Proceedings of the 8th International Conference on Educational Data Mining EDM2015, Madrid, Spain. ISBN: 978-84-606-9425-0. pp 622-623.

PP26  Kelly, K., Wang, Y., Thompson, T. & Heffernan, N.T. (2015) Defining Mastery: Knowledge Tracing Versus N- Consecutive Correct Responses, Madrid, Spain. In the Proceedings of the 8th International Conference on Educational Data Mining EDM2015, Madrid, Spain. ISBN: 978-84-606-9425-0, 630-631.

PP25 Kelly, K., Arroyo, I., & Heffernan, N. (2013). Using ITS Generated Data to Predict Standardized Test Scores. In S. D’Mello, R. Calvo, & A. Olney (Eds.) Proceedings of the 6th International Conference on Educational Data Mining (EDM2013). Memphis, TN, 322-323.

PP24 Duong, H., Zhu, L., Wang,Y. and Neil Heffernan. (2013). A prediction model that uses the sequence of attempts and hints to better predict knowledge: “Better to attempt the problem first, rather than ask for a hint.” In S. D’Mello, R. Calvo, & A. Olney (Eds.) Proceedings of the 6th International Conference on Educational Data Mining (EDM2013). Memphis, TN, 316-317.

PP23 Adjei, S., Salehizadeh, S. M. A., Wang, Y., & Heffernan, N.T. (2013). Do students really learn an equal amount independent of whether they get an item correct or wrong? In S. D’Mello, R. Calvo, & A. Olney (Eds.) Proceedings of the 6th International Conference on Educational Data Mining (EDM2013). Memphis, TN, 304-305.

PP22 Wang, Y., & Heffernan, N. T. (2013). A Comparison of Two Different Methods to Individualize Students and Skills. In Lane, Yacef, Mostow & Pavlik (Eds.) The Artificial Intelligence in Education Conference. Springer-Verlag, 836-840

PP21 Heffernan, N. T., Heffernan, C. L., Dietz, K., Soffer, D. A., Goldman, S. R., & Pellegrino, J. W. (September, 2012). Spacing Practice, Assessment and Feedback to Promote Learning and Retention. Presented at the Scientific Research on Educational Effectiveness Fall 2012 Conference in Washington, D.C.

PP20 Wang, Y. & Heffernan, N. (2011). Towards Modeling Forgetting and Relearning in ITS: Preliminary Analysis of ARRS Data. In Pechenizkiy, M., Calders, T., Conati, C., Ventura, S., Romero, C., and Stamper, J. (Eds.) Proceedings of the 4th International Conference on Educational Data Mining, 351-352. 

PP19 Goldstein, A., Baker, R. & Heffernan, N. (2010). Pinpointing Learning Moments; A finer grain P(J) model In Baker, R.S.J.d., Merceron, A., Pavlik, P.I. Jr. (Eds.) Proceedings of the 3rd International Conference on Educational Data Mining, 289-290. (Related to CP31) 

PP18 Wang, Y., Heffernan, N. & Beck, J. (2010). Representing Student Performance with Partial Credit. In Baker, R.S.J.d., Merceron, A., Pavlik, P.I. Jr. (Eds.) Proceedings of the 3rd International Conference on Educational Data Mining, 335-336.   

PP17 Rai, D., Beck, J., & Heffernan, N. (2010). Mily’s World: A Coordinate Geometry Learning Environment with Game-Like Properties. In Aleven, V., Kay, J & Mostow, J. (Eds) Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS2010) Part 2. Springer, 399-401. 

PP16 Feng, M. & Heffernan, N. T. (2010). Can We Get Better Assessment From a Tutoring System Compared to Traditional Paper Testing? Can We Have Our Cake (Better Assessment) and Eat It Too (Student Leaning During the Test)? In Aleven, V., Kay, J & Mostow, J. (Eds) Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS2010) Part 2. Springer, 309-311. (Lead to CP38) 

PP15 Feng, M., Heffernan, N., Koedinger, K. (2010). Using Data Mining Findings to Aid Searching for Better Skill Models. In Aleven, V., Kay, J & Mostow, J. (Eds) Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS2010) Part 2 Springer, 312-314. 

PP14 Patvarczki, J., Mani, M. & Heffernan, N. T. (2009). Performance driven database design for scalable web applications. In J. Grundspenkis, T. Morzy & G. Vossen (Eds) Advances in Databases and Information Systems Springer-Verlag: Berlin, 43-58 

PP13 Shrestha, P., Wei, X., Maharjan, A., Razzaq, L., Heffernan, N.T., & Heffernan, C., (2009). Are Worked Examples an Effective Feedback Mechanism During Problem Solving? In N. A. Taatgen & H. van Rijn (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society, 1294-1299.

PP12 Kim, R, Weitz, R., Heffernan, N. & Krach, N. (2009). Tutored Problem Solving vs. “Pure”: Worked Examples In N. A. Taatgen & H. van Rijn (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society, 3121-3126). Austin, TX: Cognitive Science Society.

PP11 Razzaq, L., Heffernan, N.T. (2008). Towards Designing a User-Adaptive Web-Based E-Learning System. In Mary Czerwinski, Arnold M. Lund, Desney S. Tan (Eds.): Extended Abstracts Proceedings of the 2008 Conference on Human Factors in Computing Systems, 3525-3530. Florence, Italy: ACM 2008.

PP10 Patvarczki, J., Almeida, J. F., Beck, J. E., & Heffernan, N. T. (2008). Lessons Learned from Scaling Up a Web-Based Intelligent Tutoring System. In Woolf & Aimeur (Eds.) Proceeding of the 9th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin.Volume 5091, 766-770.

PP9 Guo, Y., Heffernan, N. T., & Beck, J. E. (2008). Trying to Reduce Bottom-out hinting: Will telling student how many hints they have left help?. In Woolf & Aimeur (Eds.) Proceeding of the 9th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. Volume 5098, 774-778.

PP8 Pardos, Z., Feng, M. & Heffernan, N. T. & Heffernan-Lindquist, C. (2007). Analyzing fine-grained skill models using bayesian and mixed effect methods. In Luckin & Koedinger (Eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press, 626-628. (Based on W17)

PP7 Weitz, R., Heffernan, N. T., Kodaganallur, V. & Rosenthal, D. (2007). The distribution of student errors across schools: An initial study. In Luckin & Koedinger (Eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press. pp 671-673.

PP6 Kardian, K. & Heffernan, N.T. (2006). Knowledge engineering for intelligent tutoring systems: Assessing semi-automatic skill encoding methods. In Ikeda, Ashley & Chan (Eds.) Proceedings of the Eight International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin, 735-737. A longer version was published as WPI-CS-TR-06-06 (pdf). 

PP5 Walonoski, J. & Heffernan, N. (2006b). Prevention of off-task gaming behavior in intelligent tutoring systems. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. 2006, LNCS 4053. Springer-Verlag: Berlin, 722-724. [Winner of the Best 3-page Paper Award out of 40 such papers]. A longer version is here.

PP4 Nuzzo-Jones, G., Walonoski, J.A., Heffernan, N.T. & Livak, T. (2005). The eXtensible tutor architecture: A new foundation for ITS. In Looi, McCalla, Bredeweg, & Breuk http://www.lcc.uma.es/~eva/waswbe05/papers/nuzzo.pdf er (Eds.) Proceedings of the 12th Artificial Intelligence in Education, 555-562. Amsterdam: ISO Press, 902-904. (Based on W11)

PP3 Turner, T.E., Macasek, M.A., Nuzzo-Jones, G., Heffernan, N..T & Koedinger, K. (2005). The Assistment builder: A rapid development tool for ITS. In Looi, McCalla, Bredeweg, & Breuker (Eds.) Proceedings of the 12th Artificial Intelligence in Education, Amsterdam: ISO Press, 929-931. A longer version appears in Heffernan et al. 2006. (Based on W10)

PP2 Koedinger, K. R., Aleven, V., & Heffernan, N. T. (2003). Toward a rapid development environment for cognitive tutors. In F. Verdejo and U. Hoppe (Eds) 11th International Conference Artificial Intelligence in Education. Sydney, Australia. IOS Press, 455-457.

PP1 Razzaq, L. & Heffernan, N. T (2004). Tutorial dialog in an equation solving intelligent tutoring system. In J.C. Lester, R.M. Vicari, & F. Parguacu (Eds.) Proceedings of 7th Annual International Intelligent Tutoring Systems Conference, Berlin: Springer-Verlag, 851-853. [Winner of the Best Poster Award].

Workshop Papers

WP32  Wang, A., Prihar, E., & Heffernan, N. (2023)  Assessing the Quality of Large Language Models in Generating Mathematics Explanations. Presented at the The Fourth Annual Workshop on A/B Testing and Platform-Enabled Learning Research held at the Learning @ Scale Conference.  PDF

WP31 Haim, A., Shaw, S.T., & Heffernan, N. (2023). How to Open Science: Promoting Principles and Reproducibility Practices within the Learning Analytics Community. The 13th International Learning Analytics and Knowledge Conference, March 13-17, 2023. Arlington, TX. Submitted Paper. Final Paper. 

WP30 Sales, A., Prihar, E., Gagnon-Bartsch, J., Gurung, A & Heffernan, N. (2022). The More Powerful A/B Testing using Auxiliary Data and Deep Learning. Conference on Digital Experimentation (CODE 2022). 

WP 29 Syed, M., Prihar, E., Haim, A., Sales, A., Shaw, S., & Heffernan, N. (2021). Common Interests and Trends in Online Educational Experiments [Paper presentation]. Conference on Digital Experimentation (MIT CODE), Cambridge, Massachusetts. Paper available here.  

WP 28 Shen, J. T., Yamashita, M., Prihar, E., Heffernan, N., Wu, X., Graff, B., & Lee, D. (2021). MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education. In NeurIPS 2021 Math AI for Education Workshop. Best Paper Winner

WP 27 Sales, A., Prihar, E., Heffernan, N. (2021). Causal Inference in Educational Data Mining. (2021). Part of Educational Data Mining 2021. Blinded Review Copy.  

WP26 Choi, H., Brooks, C., Hayward, C., Kitto, K., Gasevic, D., Pardo, A., Winne, P., and Heffernan, N. (2021). Engineering Learning Analytics Technology Environments (ELATE): Understand iteration between data and theory, and design and deployment. (LAK). blinded copy.  

WP25 Botelho, A. F., Erickson, J. A., Alphonsus, A. G., & Heffernan, N. T. (2020, March) Providing Directed Feedback Through QUICK-Comments. In the 10th International Conference on Learning Analytics and Knowledge (LAK) Workshop on Learning Analytic Services to Support Personalized Learning and Assessment at Scale, Online Workshop. PDF. Slides.

WP24 Doroudi, S., Williams, J., Kim, J., Patikorn, T., Ostrow, K., Selent, D., Heffernan, N. T., Hills, T., & Rosé, C. (2018). Crowdsourcing and Education: Towards a Theory and Praxis of Learnersourcing. In Kay, J. and Luckin, R. (Eds.) Rethinking Learning in the Digital Age: Making the Learning Sciences Count, 13th International Conference of the Learning Sciences (ICLS) 2018, Volume 2. London, UK: International Society of the Learning Sciences. ICLS-Link

WP23 Wilson, K., Xiong, X., Khajah, M., Lindsey, R. V., Zhao, S., Karklin, K., Van Inwegen, E., Han, B., Ekanadham, C., Beck, J., Heffernan, N., & Mozer, M., (2016) Estimating student proficiency: Deep learning is not the panacea. Submission to the NIPS 2016 Workshop on Machine Learning for Education.

WP22  Williams, J. J., Ostrow, K., Xiong, X., Glassman, E., Kim, J., Maldonado, S. G., Reich, J., & Heffernan, N. (2015). Using and Designing Platforms for In Vivo Educational Experiments. Proceedings of the Second ACM Conference on Learning@Scale. 

WP21 Williams, J. J.,  Schultz, S., & Heffernan, N. T. (2015) Adaptively Personalizing Instruction through Collaborative Development of MOOClets by Instructors, Education, Psychology and Machine Learning Researchers. Learning with MOOCs II workshop that will be held at Teachers College, Columbia University on October 2-3, 2015.  There were two peer reviews but an unpublished acceptance rates so I listed down in this section.

WP20 Patvarczki, J., Almeida, S., Beck, J. &  Heffernan, N. (2008). Lessons Learned from Scaling Up a Web-Based Intelligent Tutoring System. Lecture Notes in Computer Science, Intelligent Tutoring Systems, 5091, 766-770. 

WP19 Razzaq, L., Heffernan, N.T. (2008). Towards Designing a User-Adaptive Web-Based E-Learning System. In Mary Czerwinski, Arnold M. Lund, Desney S. Tan (Eds.): Extended Abstracts Proceedings of the 2008 Conference on Human Factors in Computing Systems, 3525-3530. Florence, Italy.

WP18 Pardos, Z., Heffernan, N. T., Anderson, B., & Heffernan-Lindquist, C. (2007). The effect of model granularity on student performance prediction using Bayesian networks. The Educational Data Mining Workshop held at the International User Modeling Conference 2007. Corfu, Greece. (Work later led to CP18)

WP17 Pardos, Z., Feng, M. Heffernan, N. T., Heffernan-Lindquist, C. & Ruiz, C. (2007). Analyzing fine-grained skill models using Bayesian and mixed effect methods. In the Educational Data Mining Workshop held at the 13th Conference on Artificial Intelligence in Education. This is a longer version of CP18.

WP16 Lloyd, N., Heffernan, N. & Ruiz, C. (2007). Predicting student engagement in intelligent tutoring systems using teacher expert knowledge. In the Educational DataMining Workshop held at the 13th Conference on Artificial Intelligence in Education.

WP15 Feng, M., Heffernan, N. T., Mani, M., & Heffernan, C. (2006). Using mixed-effects modeling to compare different grain-sized skill models. In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press, 57-66. Technical Report WS-06-05. (Work later led to PP8)

WP14 Pardos, Z. A., Heffernan, N. T., Anderson, B., & Heffernan C. (2006). Using fine-grained skill models to fit student performance with Bayesian networks. Workshop in Educational Data Mining held at the Eighth International Conference on Intelligent Tutoring Systems. Taiwan. 2006. (Work later led to PP8)

WP13 Feng, M., Heffernan, N. T., & Koedinger, K. R. (2005). Looking for sources of error in predicting student's knowledge. In Beck. J. (Eds). Educational Data Mining: Papers from the 2005 AAAI Workshop. Menlo Park, California: AAAI Press, 54-61. Technical Report WS-05-02.

WP12 Feng, M., & Heffernan, N. (2005). Informing teachers live about student learning: Reporting in the Assistment system. Workshop on Usage Analysis in Learning Systems held at the 12th International Conference on Artificial Intelligence in Education. Amsterdam. (Work later led to J3 and J5)

WP11 Nuzzo-Jones, G., Walonoski, J.A., Heffernan, N.T. & Livak, T. (2005). The eXtensible tutor architecture: A new foundation for ITS. Workshop on Adaptive Systems for Web-Based Education: Tools and Reusability held at the 12th Annual Conference on Artificial Intelligence in Education. Amsterdam, 1-7. (Work later led to BC1 and PP4)

WP10 Turner, T.E., Macasek, M.A., Nuzzo-Jones, G., Heffernan, N..T & Koedinger, K. (2005). The Assistment builder: A Rapid development tool for ITS. In a workshop on Adaptive Systems for Web-Based Education: Tools and Reusability, held at the 12th Annual Conference on Artificial Intelligence in Education Amsterdam. (Work later led to BC1, PP4 and CP14)

WP9   Freyberger, J., Heffernan, N., & Ruiz, C. (2004). Using association rules to guide a search for best fitting transfer models of student learning. In Beck, Baker, Corbett, Kay, Litman, Mitrovic & Rigger (Eds.) Workshop on Analyzing Student-Tutor Interaction Logs to Improve Educational Outcomes. Held at the 7th Annual Intelligent Tutoring Systems Conference, Maceio, Brazil. Lecture Notes in Computer Science. 

WP8   Livak, T., Heffernan, N. T., Moyer, D. (2004). Using cognitive models for computer generated forces and human tutoring.13th Annual Conference on (BRIMS) Behavior Representation in Modeling and Simulation. Simulation Interoperability Standards Organization. Arlington, VA.

WP7   Razzaq, L. & Heffernan, N. T (2004). Tutorial dialog in an equation solving intelligent tutoring system. Workshop on “Dialog-based Intelligent Tutoring Systems: State of the art and new research directions” at the 7th Annual Intelligent Tutoring Systems Conference, Maceio, Brazil, 33-42.

WP6   Koedinger, K. R., Aleven, V., & Heffernan, N. T. (2003). Toward a rapid development environment for cognitive tutors. The 12th Annual Conference on Behavior Representation in Modeling and Simulation. Simulation Interoperability Standards Organization. (Work later led to CP7)

WP5   Heffernan, N. T., (2002). Web-based evaluation showing both motivational and cognitive benefits of the Ms. Lindquist tutor. SIGdial endorsed Workshop on Empirical Methods for Tutorial Dialogue Systems which was part of the International Conference on Intelligent Tutoring System 2002,1-8. Also appeared in a NSF-DFG sponsored workshop on Collaboration between German and American researchers in instructional technology. Tampa, Florida, May 5-7, 2002. (Work later led to CP8)

WP4   Heffernan, N. T, Koedinger, K. (2001). The design and formative analysis of a dialog-based tutor. Workshop on Tutorial Dialogue Systems held as part of the 2001 Artificial Intelligence in Education, 23-34. (Work later led to CP8)

WP3   Heffernan, N. T., & Koedinger, K. R. (2000). Building a 3rd generation ITS for symbolization: Adding a tutorial model with multiple tutorial strategies. Workshop entitled “Learning Algebra with the computer, a transdisciplinary workshop.” Held at Intelligent Tutoring Systems 2000 Conference, 12-22. Lecture Notes in Computer Science 1839, Berlin: Springer. (Work later led to CP8)

WP2   Heffernan, N. T. (2000). Adding a cognitive model of human tutor to an intelligent tutoring systems. Intelligent Tutoring System Conference- Young Researchers Track. (Work later led to D1)

WP1   Heffernan, N. T. (1998). Intelligent tutoring systems have forgotten the tutor: Adding a cognitive model of human tutors. Abstract at the 1998 Computer Human Interaction Conference’s Doctoral Consortium. (Work later led to D1)

Unknown Acceptance Rate

U18 Wei, X., Wortman, A., Cheng, L., Heffernan, N., Heffernan, C., Murphy, A., Zepeda, C., Motz, B., Jankowski, H., & Roschelle, J. (2024, March). Language and mathematics learning: A comparative study of digital learning platforms. Digital Promise. doi.org/10.51388/20.500.12265/206

U17 Prihar, E., Moore, A. & Heffernan, N. (2022) Identifying Explanations Within Student-Tutor Chat Logs. DC paper at EDM2022. View

U16 Baral, S. (2022). Improving Automated Assessment and Feedback for Student Open-Responses in Mathematics. EDM 2022 Doctoral Consortium. 

U16 Prihar, E., Moore, A., Heffernan, N. (2022). Identifying Explanations Within Student-Tutor Chat Logs. EDM 2022 Doctoral Consortium. 

U15 Gurung, A., Heffernan, N. (2022). Exploring Fairness in Automated Grading and Feedback Generation of Open-Response Math Problems. AIED 2022 Doctoral Consortium. 

U14 Haim, A., Prihar, E., & Heffernan, N. (2022). Toward Improving Effectiveness of Crowdsourced, On-Demand Assistance From Educators in Online Learning Platforms. AIED 2022 Doctoral Consortium.

U13 Singla, A., Rafferty, A, Radanovic, G & Heffernan, N. (2021) Reinforcement Learning for Education: Opportunities and Challenges.  An overview of what happened at a RL4ED.org 2021 EDM Workshop. https://arxiv.org/abs/2107.08828

U12 Goldstein, D.S., Heffernan, C., Heffernan, N.T., Pellegrino, J.W., Goldman, S.R., & Stoelinga, T.M. (April 2016). Mapping skills and knowledge in the Connected Mathematics Project 2 Curriculum. Poster presented at Annual Meeting of the American Educational Research Association, Washington, DC. 

U11 Goldstein, D.S., Pellegrino, J.W., Goldman, S.R., Stoelinga, T.M., Heffernan, N.T., & Heffernan C. (April 2016). Improving mathematical learning outcomes through applying principles of spaced practice and assessment with feedback. Poster presented at Annual Meeting of the American Educational Research Association, Washington, DC.

U10 McGuire, P., Logue, M., Mason, C., Tu, S., Heffernan, C., Heffernan, N., Ostrow, K. & Li, Y. (2016, accepted). To See or Not To See: Putting Image-Based Feedback in Question. Interactive lecture at the International Society for Technology in Education Conference. Denver, CO. 

U9 Trivedi, S.,, Pardos, Zachary A. , &  Heffernan, N. T. (2023) The Utility of Clustering in Prediction Tasks. https://doi.org/10.48550/arXiv.1509.06163 

U8 Williams, J. J., Krause, M., Paritosh, P., Whitehill, J., Reich, J., Kim, J., Mitros, P., Heffernan, N., & Keegan, B. C. (2015). Connecting Collaborative & Crowd Work with Online Education. Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing, 313-318. 

U7 Williams, J., J., Li, N., Kim, J., Whitehill, J., Maldonado, S., Pechenizkiy, M., Chu, L., & Heffernan, N. (2014). MOOClets: A Framework for Improving Online Education through Experimental Comparison and Personalization of Modules (Working Paper No. 2523265). The Social Science Research Network.

U6 Williams, J. J., Maldonado, S., Williams, B. A., Rutherford-Quach, S., & Heffernan, N. (2015). How can digital online educational resources be used to bridge experimental research and practical applications? Embedding In Vivo Experiments in “MOOClets” . Paper presented at the Spring 2015 Conference of the Society for Research on Educational Effectiveness, Washington, D. C.

U5 Williams, J., J., Li, N., Kim, J., Whitehill, J., Maldonado, S., Pechenizkiy, M., Chu, L., & Heffernan, N. (2015). MOOClets: A Framework for Improving Online Education through Experimental Comparison and Personalization of Modules (Working Paper No. 2523265). The Social Science Research Network: 

U4 Kelly, K., Heffernan, N., Heffernan, C., Goldman, S., Pellegrino, J., & Soffer-Goldstein, D. (2014). Improving student learning in math through web-based homework review. In Liljedahl, P., Nicol, C., Oesterle, S., & Allan, D. (Eds.). (2014). Proceedings of the Joint Meeting of PME 38 and PME-NA 36 (Vol. 3). Vancouver, Canada: PME, 417-424.

U3 Pellegrino, J., Goldman, S., Soffer-Goldstein, D., Stoelinga, T., Heffernan, N., & Heffernan, C. (2014).  Technology Enabled Assessment:Adapting to the Needs of Students and Teachers. American Educational Research Association (AERA 2014) Conference.  

U2  Soffer,-Goldstein, D., Das, V., Pellegrino, J., Goldman, S., Heffernan, N., Heffernan, C., & Dietz, K. (2014). Improving Long-term Retention of Mathematical Knowledge through Automatic Reassessment and Relearning. American Educational Research Association (AERA 2014) Conference. Division C - Learning and Instruction / Section 1c: Mathematics. (peer reviewed but unknown rate Nominated for the Best Poster of the Session. The paper and data are here :   https://sites.google.com/site/assistmentsdata/arrs 

U1 Heffernan, N., Heffernan, C., Dietz, K., Soffer, D., Pellegrino, J. W., Goldman, S. R. &  Dailey, M. (2012). Cognitively-Based Instructional Design Principles: A Technology for Testing their Applicability via Within-Classroom Randomized Experiments. AERA 2012.

Dissertation

Heffernan, N. T (2001). Intelligent tutoring systems have forgotten the tutor: Adding a cognitive model of human tutors Dissertation. Computer Science Department, School of Computer Science, Carnegie Mellon University. Technical Report CMU-CS-01-127. (Pieces published as CP1, CP8, CP5, CP4, and CP3)

Workshops & Tutorials I Helped Organize

WP18 Haim, A., Shaw, S.T., & Heffernan, N. (2023d, March 13). How to Open Science: Promoting Principles and Reproducibility Practices within the Learning Analytics Community. The 13th International Learning Analytics and Knowledge Conference, March 13-17, 2023. Arlington, TX. Final Paper  https://osf.io/kyxba/  Its in this

WP17 Haim, A., Shaw, S.T., & Heffernan, N. (2023e, July 3rd, ): How to Open Science: Promoting Principles and Reproducibility Practices Within the Artificial Intelligence in Education Community https://doi.org/10.1007/978-3-031-36336-8_11 

WP16 Haim, A., Shaw, S.T., & Heffernan, N. (2023f, July 14rd): How to Open Science: Promoting Principles and Reproducibility Practices with the Educational Data Mining Community. Pages 582-584. https://doi.org/10.5281/zenodo.8115776 

WP15 Haim, A., Shaw, S.T., & Heffernan, N. (2023g, July 23) How to Open Science: Promoting Principles and Reproducibility Practices within the Learning @ Scale Community. Pages 248-250. https://doi.org/10.1145/3573051.3593398

WP14 LAK (March 13-17, 2023): Participatory Co-Design of Platform-Embedded Learning Experiments. Fancsali, S., Ritter, S., Malick, D.B., Motz, B., Heffernan, N., Baker, R., Kizilcec, R., Roschelle, J., & McNarmara, D. The 13th International Learning Analytics and Knowledge Conference, March 13-17, 2023. Arlington, TX. Final Paper.

WS13 L@S (June 1-3, 2022): Third Annual Workshop on A/B Testing and Platform-Enabled Learning Research by Ritter, S., Heffernan, N., Williams, J.J., Lomas, D., Motz, B., Mallick, D.B., Bicknell, K., McNamara, D., Kizilcec, R.F., Roschelle, J., Baraniuk, R., & Baker, R. L. L@S, June 1-3, 2022. PDF. https://doi.org/10.1145/3491140.3528288

WS12 AAAI (March 1, 2022): RL4ED Workshop on Reinforcement Learning for Education by Heffernan, N., Lan, A., Rafferty, A., & Singla, A.

WS11 EDM (June 29-July 2, 2021): Reinforcement Learning for Education: Opportunities and Challenges by Singla, A., Rafferty, A, Radanovic, G & Heffernan, N. https://arxiv.org/abs/2107.08828

WS10  EDM (June 29-July 2, 2021): Causal Inference in Educational Data Mining Causal inference in Educational Data Mining by Sales, A., Prihar, E., & Heffernan, N. https://sites.google.com/umich.edu/causaledm21

WS9 L@S (June 22-25, 2021): Second Workshop on Educational A/B Testing at Scale by Ritter, S., Heffernan, N., Williams, J. J., Lomas, D., & Bicknell, K. https://doi.org/10.1145/3430895.3460876

WS8  LAK (April 12-16, 2021): ELATE: Engineering Learning Analytics Technology Environments: Understanding iteration between data and theory, and design and deployment by Choi, H., Brooks, K., Hayward, C., Heffernan, N., Gasevic, D., Kitto, K., Pardo, A., & Winne, P. https://cic.uts.edu.au/events/lak21

WS7  AAAI (Feb. 2-9, 2021): TIPCE 2021 Imagining Post-COVID Education with AI program committee member

WS6  NeurIPS (Dec. 6-12, 2020): Minimizing Bias in Machine Learning

WS5 NeurIPS (Dec. 16-20, 2020): Advances and Opportunities: Machine Learning for Education by Garg, K., Heffernan, N., & Meyers, K. https://nips.cc/Conferences/2020/Schedule?showEvent=16104 

WS4  SIGKDD (Aug. 2020): Recent Advances in Multimodal Educational Data Mining in K-12 Education in KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Pages 3549–3550. https://doi.org/10.1145/3394486.3406471 

WS3 L@S (Aug. 12, 2020): Artificial Intelligence for Video-based Learning at Scale. Was asked to give a keynote talk for this workshop. Pages 215–217. https://dl.acm.org/doi/abs/10.1145/3386527.3405937 

WS2  L@S (Aug. 12-14, 2020): Educational A/B Testing at Scale with over 100 participants by Ritter, S., Heffernan, N., Williams, J. J., Settles, B., Grimaldi, P., & Lomas, D. Pages 219–220. https://dl.acm.org/doi/abs/10.1145/3386527.3405933

WS1  Heffernan was asked to give the keynote to the Artificial Intelligence in Education 2020 Conference where he explained ASSISTments and his vision for crowdsourcing. https://www.youtube.com/watch?v=S-AydzWsjeU&feature=youtu.be