IES Contribution

ASSISTments Development supported by IES / DOE

In 2003, an IES grant (R305K030140: 2003-2007) funded the creation of ASSISTments, an online tool to help teachers and students. With this tool, WPI personnel and Worcester Public School teachers could input questions with associated feedback and deliver the content to students. The first problems in the system were simply Massachusetts’ released high-stakes state math test questions. WPI students entered these questions and wrote detailed feedback for each question following the advice of the experienced teachers. Teachers were able to choose what to assign from the library of premade content or they could write their own questions. Reports provided teachers with information on 1) which questions were hardest for their students and 2) what common wrong answers were being submitted by students. This information gave the teachers data to use to initiate class discussions around the completed assignments. 

To give something for teachers to assign for repetitive practice Heffernan won his second IES grant (R305A070440: 2007-2011) to create and research sets of  “skill builders.” WPI created 200 skill builder problem sets for middle school math topics. Each skill builder gives students practice until they reach some initial threshold of performance. Students who do well on a skill builder finish quickly, but students who are not proficient in the skill will be given practice and hints until they get three correct in a row. Grants from NSF allowed Heffernan to extend the idea of mastery learning to a system called Automatic Reassessment and Relearning (ARRS). After a student completes questions to show skill proficiency,  one week later the students are automatically reassessed on the skills assigned by their teacher. If they have forgotten the skill, they received remediation. In addition to this extension grant on the idea of skill builders initially funded by IES, NSF has funded ASSISTments eight times to support the development of and research with ASSISTments.

All of this work combined into what was ASSISTments in 2012 was tested as an effective online homework intervention by the IES Efficacy grant (R305A120125: 2012-2016). In order to conduct the study, the team took every commonly used 7th grade textbook in the State of Maine and added all the answers to textbook homework problems into ASSISTments. The system was also made to work offline so a student could work on the school bus, for example, away from wifi. The project assessed the role of immediate feedback to both teachers and students with a particular focus on the need for low-income students to get smarter homework. In 2016 the results of that Efficacy Trial showed that 1) teachers changed their homework reviewing behavior, 2) students learned reliably more and that difference was an extra three quarters of a year of schooling, and 3) Using ASSISTments reliably closed achievement gaps contrary to current research that “draws attentions to the issue of whether computerized learning might contribute to the achievement gap between students with different achievement levels” (Steenbergen-Hu & Cooper, 2013).  

The study does not tease out why the tool may have worked so well, but Heffernan and his wife Cristina have a hypothesis. “Cristina and I were both school teachers and we think our system works well because we designed it to be used by teachers, rather than to replace teachers. Many education technology products put the computer in charge, and try to promote the students through the curriculum at just the right pace, fast for some students and slow for others. When this happens the teacher who is an important factor in motivating students, is no longer able to track the data because each student is in a different place. We let teachers give out short cycle adaptive assignments while remaining in charge of the pacing. Each morning the teacher checks in and uses the data to support decisions about the next lesson. We recognize that this is a different mindset from the push to highly personalized products. We have created a product that is highly personalized to the teacher not the student and we let the teacher use the data to  individualize their feedback to their students and to inform the social part of learning that is their daily class.” 

Now that the Efficacy Trial proves that ASSISTments works, where is ASSISTments going next? He points out that of the 80+ teachers that were in the Efficacy Trial, one teacher wrote a feedback message for every problem he assigned from his textbook. Heffernan wants to make it possible for other teachers to use that feedback. In other words he would like to crowdsource one or more teachers' feedback and give it to other teachers using the same textbook. Heffernan says, “My life’s goal is to be like Jimmy Wales who created Wikipedia that allows people to write encyclopedia entries, but I want to do that for feedback on every math and science questions given to students in K12 in America, that means letting teachers write the feedback and letting other teachers find it and use it.”

In 2023, the results of the IES Replication grant found the same effect but in North Carolina. 

When asked why ASSISTments is free, he responds, “One day I  had a seizure and was diagnosed with a brain tumor and I was told I had 2-3 years to live. I did chemo and radiation and surgery for a year. Luckily the doctors were wrong and I have beaten my cancer. The experience changed me, my wife and I decided that our goal in life had to be to make a difference. I am glad Worcester Polytechnic Institute has just donated two staff members to the project for the next two years to keep it growing. Last year alone 12 million problems were solved by 50,000 students. Next year looks to be even bigger. And like Wikipedia, we have to be free if we are going to ask teachers to share in this endeavor in our crowdsourcing project.”

Here is a partial listing of grants that have supported the growth and development of ASSISTments. 

U.S. Department of Education has funded Heffernan from the Institute for Education Sciences 

Heffernan received eight grants from the NSF that are described here

Heffernan has been funded three times by the US Dept of Education's GAANN program

ONR has funded Heffernan five times to build better tools to author intelligent tutoring systems

More Recently A different arm of the US Dept of Ed called EIR  has funded ASSISTments. 

New Notes