There is a crisis in science where many papers are difficult or impossible to reproducible. The US Department of Education's Institute for Education Science have adopted the Standards for Excellence in Education Research (SEER) and those call for openness in education such as making data available (when possible), making materials available, and preregistering. These ideas are also endorsed by the Center for Open Science through badges.
I am proud of the work my student Aaron Haim, who, along with Prof. Stacy Shaw, has pursued an open science agenda.
He has written 3 papers in this area. In those paper we report on the open science practices of three of the major conferences in this area (LAK, EDM, and L@S). In each paper he pulls down the ~100 of the most recent papers from that venue and reads them for open science. The results show that 1) only about 5% qualify for the Open Data badge, 2) about 5% qualify for the Open Materials badge, 3) 2-4% qualify for the Preregistration badge, and 4) only 2-3% of the papers give out data and code such that they could reproduce the results in the paper. Over all these are sad findings for our field, as science is best when it can be replicated.
Feeling that maybe many researchers just don't know how to do open science, Aaron Haim is also running workshops on how to do open science at L@S, LAK, EDM and AIED.
Finally, we are proud to have helped EDM adopt the OSF Open Badges. In 2023, authors were given a chance, when submitting their paper, to choose which OSF badges they thought they qualified for. Hopefully, this will signal to authors to make their science work open, so we hope to look in 2024 to see if more papers are open.