Service

When most academics think of “service” they think reviewing, running a scientific society, and/or being Program Chair. I have done all three, which are detailed below. However, I am also proud of some types of more unique services that involve ASSISTments as a shared scientific instrument. They include 1) letting others use ASSISTments to run studies, 2) giving out ASSISTments data for research by others, and 3) running competitions to predict ASSISTments data.


I Provide ASSISTments as a Shared Scientific Instrument to Other Researchers

The following researchers used the ASSISTments infrastructure to run studies that support their research objectives. With the support of funding from an NSF award, I was able to help these external researchers run randomized controlled trials within the ASSISTments platform. The platform is open for new researchers at http://www.assistmentstestbed.org.

 

RCT1 Fyfe, E. (2016) Providing feedback on computer-based algebra homework in middle-school classrooms. Computers in Human Behavior 63. 568-574

RCT2 Koedinger, K. & McLaughlin, E. (2016) Closing the Loop with Quantitative Cognitive Task Analysis. In Barnes, Chi & Feng (eds) The 9th International Conference on Educational Data Mining. Pp 412-417.

RCT3 McGuire, P., Tu, S., Logue., M., Mason, C., Ostrow, K. (2017) Counterintuitive effects of online feedback in middle school math: results from a randomized controlled trial in ASSISTments. Educational Media International. 54:3, 231-244, DOI: 10.1080/09523987.2017.1384161

RCT4 Clinton, V., Walkington, C., & Spark (2018) The effect of modifying the language of mathematics story problems on problem-solving measures. To be presented at The 2019 Annual Meeting of the American Educational Research Association, Toronto, Canada.

RCT5 Jiang, Yang et al. (In preparation)


Research Papers Utilizing ASSISTments Data for Open Science

As a public adopter of open science, I require that my Ph.D. students, if they are going to use ASSISTments data in a paper, must provide a URL link to the data they used in their paper, and whatever code they wrote to get the results they report in the paper (e.g., Python code, or MathLab code). This allows other researchers to try to improve upon our methods. I am proud to report I am aware of 23 papers in which other academics have used our publicly available datasets for their analysis. Their articles can be found here.


I Have Run a Competition to Engage Folks in Educational Data Mining

I was funded with an NSF BD Spoke grant to engage and educate people on BigData in Education. With my colleagues, I ran a competition using some ASSISTments data. 75 researchers participated. We gave out some awards, and hosted a special issue of the Journal on Educational Data Mining for winners to submit results.

International Reviewing

When I started publishing my home conference was the Artificial Intelligence in Education Conference; I have reviewed for the journal and conference since 2002. I was elected to serve as program co-chair in 2015. I have been elected to the steering committee twice. This conference is stringently reviewed and has been averaging an acceptance rate of about 30% for the last 15 years.

Though this conference has been invaluable, I, along with a few of my colleagues, saw a need for a more specialized group and created the International Educational Data Mining Society to both manage a new conference and publish a journal. I and 6 others created the conference and ran it for the first three years before turning it into an open democratic scientific society. I was elected and served two terms on the 7-person steering committee. I have a great fondness for this group, and what we have accomplished, particularly how we use open science policies. This means the papers are all free (i.e., Creative Commons) as we refused to partner with a publisher that would only inflate the cost of the proceedings.

I also do journal reviewing for TOCHI, SREE, and other journals when relevant to my interest areas.  

  • Reviewed for SREE, CHI, AIED, AERA & EDM (2018).

  • Elected to the Executive Committee for the Society for Artificial Intelligence in Education (August 2017).

  • Reviewer for LAK, L@S AIED and CHI 2016-217.

  • Co-Chair of the Program Committee for AIED2015

  • Reviewer for to CHI and CSCW 2015

  • Steering Committee: The Eighth International Conference on Educational Data Mining 2015.

  • Steering Committee: The Seventh International Conference on Educational Data Mining 2014.

  • Program Committee: Intelligent Tutoring Systems Conference 2012.

  • Steering Committee: The Sixth International Conference on Educational Data Mining 2013.

  • Program Committee: Artificial Intelligent in Education Conference 2013.

  • Program Committee: User Modeling And Adaptive Personalization 2013.

  • Steering Committee: The Fifth International Conference on Educational Data Mining 2012.

  • Program Committee: Intelligent Tutoring Systems Conference 2012.

  • Program Committee: Artificial Intelligent in Education Conference 2011.

  • Steering Committee: The Fourth International Conference on Educational Data Mining 2011.

  • Program Committee: User Modeling And Adaptive Personalization 2011.

  • Steering Committee: The Third International Conference on Educational Data Mining 2010.

  • Program Committee: Intelligent Tutoring Systems Conference 2010.

  • Steering Committee: The Second International Conference on Educational Data Mining 2009.

  • Program Committee: The Second International Conference on Educational Data Mining 2009.

  • Program Committee: Artificial Intelligent in Education Conference 2009.

  • Poster Co-Chair for Artificial Intelligent in Education Conference 2009.

  • Program Committee: Intelligent Tutoring Systems Conference 2008.

  • Demonstrations: Intelligent Tutoring Systems Conference 2008.

  • Program Committee: First International Conference on Educational Data Mining 2008.

  • Senior Program Committee: International Conference on Artificial Intelligence in Education 2007.

  • Program Committee: American Association for Artificial Intelligence 2007 (AAAI'07).

  • Co-Chair of “Educational Data Mining Workshop” to be held at the International Conference on Artificial Intelligence in Education 2007.

  • Reviewed for IEEE Intelligent Systems Journal. 2007.

  • Reviewer for User Modeling and User-Adapted Interaction: The Journal of Personalization Research 2007.

  • Reviewer for Cognitive Science Conference 2007.

  • Program Committee: International Intelligent Tutoring System Conference 2006.

  • Program Committee: FLAIRS-2006 Special Track on Intelligent Tutoring Systems 2006.

  • Program Committee: International Conference on Artificial Intelligence in Education 2005.

  • Workshops Co-Chair International Conference on Artificial Intelligence in Education 2005.

  • Reviewer for International Intelligent Tutoring System Conference 2004.

  • Chair of workshop on Dialog-Based Intelligent Tutoring Systems: State of the art and new research directions held at the Intelligent Tutoring Systems Conference 2004.

  • Reviewer for International Conference on Artificial Intelligence in Education 2002.

  • Reviewed for International Journal of Computers for Mathematical Learning.

  • On the program committee for numerous workshops held in conjunction with AIED and ITS conferences.

National Service

  • I have served as an advisor to the National Assessment of Educational Progress (NAEP), otherwise known as "The Nation's Report Card." 

  • I have done NSF proposal review panels about 10 times.

  • I am an adviser to the CAST $10 million special education center.

  • I was invited to serve on the NSF/CRA CyberLearning Workshop focused on Technology-Enabled Assessment to help shape a research agenda for NSF's EHR Directorate.

Service to my home institution: Worcester Polytechnic Institution

  • The WPI Outcomes Assessment Committee (3 years)

  • The Computer Science Department: Graduate Research Committee (10 years), Colloquium Committee (2 years, 1 year as chair), Admissions (1 year)