This paper presents innovative approaches to rethinking education and peer review in the age of artificial intelligence (AI). A brief overview of the current discourse surrounding education and the peer review process is presented in addition to a discussion on how data and AI can positively impact the overall educational system and the academic peer review mechanism for quality control. Necessary policy considerations are addressed as well suggestions for areas of future research.
Introduction: Background
The history and projections of data science as the enabler of the “digital revolution” and AI is well-known and has been established as an academic discipline since the 1950s despite limited practical applications over half a century.1 In this discussion, AI is referred to as computer-based systems that can carry out tasks that usually require human intelligence. This includes two types of systems: “closed” systems that work on predetermined data sets; and “open” systems that can update their knowledge base by continuously gathering new data. The most profound recent developments driving accelerated change and the adoption of a vast array of applications arises from an exponential growth in computational power. For example, the global annual internet traffic was only 60 petabytes in 1997 but skyrocketed to around 4.2 zettabytes in 2022.2 Global interconnectedness, made possible through ubiquitous availability of low-cost sensors and devices and enabled through the ”internet of things” technologies, has again increased vastly within the last decade. Today, more than 7.7 billion mobile subscriptions have outpaced the world’s population.3 It is worth noting that the number of subscriptions from the Global South and developing countries is very small due to limited access to the necessary devices and power required to charge them. Furthermore, the availability of receivers for internet access through satellite systems in low- and middle-income countries is also a significant challenge. The explosive growth in data has opened up diverse and new opportunities on an unprecedented scale by leveraging data and AI in innovative ways for various applications. However, it is important to note that data sources are currently inherently biased and predominantly populated by high-income countries.
Numerous studies highlight the effects and usage of AI in different fields. The impacts of automation and AI on the labour market are analyzed by Daron Acemoglu and Pascual Restrepo within a framework to examine the displacement effect, where automation replaces human labour, potentially decreasing labour demand, wages and employment.4 However, Acemoglu and Restrepo also highlight the productivity effects that can balance out these negative impacts, leading to increased labour demand in areas where automation is not yet present and in other industries. This study emphasized the complex interplay between various forces shaping the labour market in the current era of rapid automation. Researchers have also discussed how generative AI tools, such as ChatGPT, can be used in natural science and engineering to answer difficult questions more rapidly. In 2023, Lukas Schulze Balhorn et al. conducted a study at Delft University of Technology that evaluated ChatGPT’s proficiency in natural science and engineering.5 The analysis involved 198 faculty members who posed 594 questions across different academic levels. The study’s findings indicated that ChatGPT generally provided “mostly correct” answers, especially for bachelor-level queries. However, its performance declined with more complex questions and tasks that required advanced skills such as critical thinking. The study also highlighted the limitations of ChatGPT’s in-depth knowledge and critical reflection, particularly at higher academic levels. Additionally, the ethical implications of using ChatGPT were discussed. In their study, Jingshan Huang and Ming Tan analyzed the role of ChatGPT in assisting scientists with crafting scientific review articles.6 They highlighted its potential to improve efficiency and quality in scientific writing. The study discussed ChatGPT’s ability to expedite the writing process, suggest improvements, provide diverse perspectives and offer particular assistance to non-native English speakers. However, the study also recognized limitations and ethical considerations, such as the risk of plagiarism and the need for human oversight. The study concludes that ChatGPT should complement human expertise, not replace it, to maintain quality and integrity in scientific communication.
The Academy
Research, defined as the creation of new knowledge and teaching, which encompasses the transfer of existing knowledge and the facilitation of learning to learn and generate new knowledge, are fundamental to the academic enterprise. Within this context, the education and peer review processes are critical for enhancing the quality and innovation of educational practices. Education is the primary building block essential for shaping societal development and profoundly impacts individual growth and national productivity and progress. Similarly, the peer review process is fundamental to research. This rigorous process plays a critical role in refining the quality of scholarly research, contributing significantly to advancing knowledge and its applications to support innovation, and improving the overall quality of life. With the emergence of AI, a remarkable opportunity exists to enhance both the educational system and the peer review process. AI holds the potential to not only elevate the quality of education and peer review, but also to address prevalent issues within these domains. By harnessing the capabilities of AI, we can unlock new efficiencies and insights, revolutionizing these critical areas of academic and societal function.
AI in Education
Many academic journal articles, conference papers and technical reports on AI in education have already been published. In their analysis, Fan Ouyang and Pengcheng Jiao describe three critical paradigms that highlight the evolving influence of AI in education: AI-directed learner-as-recipient; AI-supported learner-as-collaborator; and AI-empowered learner-as-leader.7 The first paradigm, AI-directed learner-as-recipient, portrays AI as a guiding force where learners passively receive AI-curated knowledge and direction. This model focuses on AI’s role in shaping cognitive learning through structured pathways. The second paradigm, AI-supported learner-as-collaborator, depicts AI as a supportive entity that fosters a collaborative learning environment. In this setting, learners interact with AI actively, contributing to their learning process, while AI tailors its responses to individual needs. Finally, the AI-empowered learner-as-leader paradigm places learners at the forefront of their educational journey, with AI enhancing and amplifying their learning capabilities. This approach emphasizes learner autonomy, customization and the fusion of human and AI intelligence, aiming to create an adaptive and personalized learning experience.
Yusi Teng, Jie Zhang and Ting Sun explore using a data-driven decision-making model (DDMM) that employs machine learning techniques in higher education.8 This model exemplifies an open system, as it continually updates its knowledge base by integrating new data, aligning with the previously introduced concepts of closed and open systems. The DDMM is designed to improve administrative decisions in colleges and universities by carefully analyzing student data, graduation rates and curriculum structures. The authors suggest that the DDMM can effectively enhance key performance indicators in educational institutions, such as outcome and prediction ratios, decision-making efficiency, precision, operational efficiency and data security.
Hui Luan et al. discuss the emerging trends, challenges and promising applications of big data and AI in education in their study.9 They highlight these technologies’ transformative impact on teaching and learning methodologies, such as developing personalized learning experiences and advancing precision education. Overall, the paper emphasizes the potential benefits of big data and AI in shaping the future of education. At the 2nd International Conference on AI in Education Technology in 2021, numerous research papers on AI in education were also presented.10 The conference proceedings emphasized the rapid advancements in AI) and its potential for educational applications, generating interest among the academic community. This collection covers various theoretical and practical aspects of AI in education, including technical, pedagogical, administrative and socio-cultural considerations.
Enhancing the Peer Review Process with AI
The integrity and quality of scientific publications are fundamental to academic progress, with the peer review process playing a critical role in ensuring the veracity of data and reliability of outputs. The challenge of distinguishing genuine research from the surge of fake papers highlights a pressing issue; over the past two decades, a substantial number of articles have shown strong textual similarities to works from known paper mills, indicating a widespread problem, particularly in disciplines such as biology, medicine, chemistry and materials science.11, 12 Studies show that over the past two decades, approximately 400,000 articles have established strong textual similarities to papers from known paper mills, with around 70,000 in 2022 alone mirroring the style of these dubious works.13 This proliferation of dubious research necessitates exhaustive investigations to maintain the integrity of scientific literature, underscoring the urgency for effective detection and prevention strategies.14.
AI offers promising solutions to refine and expedite the peer review process, addressing both the efficiency of operations and the quality of academic scrutiny. The application of evolutionary computation, including Cartesian Genetic Programming, presents an innovative approach to developing editorial strategies that significantly reduce the peer review timeline by up to 30 percent without necessitating additional reviewers.15 This demonstrates AI’s potential to streamline complex social systems like peer review, which is traditionally challenging to optimize.
Furthermore, the Automated Scholarly Paper Review (ASPR) concept signifies a paradigm shift towards augmenting traditional peer review with AI, aiming to mitigate common criticisms such as inefficiency and low reproducibility.16 While developing a comprehensive computerized review process faces challenges such as insufficient data, imperfect document parsing and inadequate deep logical reasoning, exploring ASPR underscores the potential for AI and traditional methods to coexist and complement each other, enhancing the academic publishing landscape.
The increasing difficulty of securing reviewers in the face of a growing submission volume brings to light the potential of automation technologies. These technologies not only assist in the administrative aspects of the review process but also play a crucial role in maintaining quality and identifying suitable reviewers.17 However, the transition toward automation must be cautiously navigated to avoid exacerbating existing disparities within the academic community. The balance between the benefits of automation and the preservation of the valuable elements of the traditional peer review system is essential, especially for supporting early career researchers and fostering scholarly communities.
In addition to streamlining the peer review process, AI’s role supports various aspects of academic publishing, including journal selection, initial quality control of submissions and reviewer assignment.18 Despite the promising utility of automation in these areas, the effectiveness of AI in comprehensively supporting the review process remains to be conclusively demonstrated. Moreover, while peer review texts and scores theoretically offer valuable insights for post-publication research assessment, their limited availability hinders systematic automation’s practical application.19
By integrating AI into the peer review process, the academic community can address the critical challenge of ensuring the integrity and quality of scientific publications. This integration not only aids in combatting the issue of fake papers through enhanced detection mechanisms — akin to those developed for plagiarism detection in student papers — but also promises to improve the efficiency and effectiveness of peer review, a cornerstone of academic scholarship.
A General Overview on AI’s Benefits in Education and the Peer Review Process
Reflecting on the literature (see endnotes 7–10), the benefits of data and AI in the educational sector can be outlined as follows:
- Several critical challenges, such as challenges for educational innovations and personalized learning7, can be effectively addressed with help of advancements in data science and AI solutions for improved educational outcomes.
- One key area is to improve curriculum design, making it more effective and relevant.
- Use of these technologies can be instrumental in improving student evaluation and selection processes.
- Integrating data and AI has greatly aided the development of innovative teaching techniques and paved the way for new pedagogical approaches in education.
- AI can provide personalized education by identifying and addressing individual learning gaps through customized content.
- AI for advancing intelligent tutoring systems can enhance the learning experience.
- AI tools are vital in developing high-quality educational materials and improving the overall content quality.
- Data and AI are essential in creating platforms that continuously augment teaching skills and knowledge among educators.
The primary advantages of utilizing data and AI in the peer review process based on the current research activities are as follows:
- AI can transform the peer review process by automating the evaluation of critical aspects such as a paper’s originality, reproducibility and contribution to research and industry. Automating these evaluations is increasingly vital, given the high volume of manuscript submissions to journals. An AI-based platform could greatly assist editors and reviewers in making these assessments more efficient.
- AI can significantly reduce the time taken for the peer review process, streamlining the workflow and accelerating publication timelines.
- Data and AI can enhance the transparency of the peer-review process, providing clearer insights into the decision-making and review strategies.
- A relatively recent development of repositories of data collected with public funds is a ready source for applying AI techniques to verify data and establish the originality of research. Such a system would enhance existing public repositories, allowing authors to not only openly share their data and methodologies, but also to have AI support in flagging when a research question has already been examined. This approach would increase transparency, encourage reproducibility and support creation of novel research questions with existing data, leading to a more trustworthy and reliable academic publishing environment.
Implementing AI in Education and Academic Peer Review: Focused Strategies and Insights
Drawing from research activities on AI in education, it is clear that a collaborative effort across various multidisciplinary domains is fundamental for effectively integrating AI into educational frameworks. Addressing ethical and algorithmic challenges, such as algorithmic bias and ensuring equitable access to tech-enhanced learning, are critical aspects that necessitate this collaboration. Policy and industry considerations must also be considered when incorporating AI into educational practices, particularly regarding data protection, privacy concerns and fostering collaborations among diverse stakeholders.
The peer review process depends on evaluating a manuscript’s originality and critical contributions, along with other crucial considerations such as ethics, image manipulation, plagiarism, methodology, results interpretation, manuscript readability and accessibility, data accessibility and language.20
The literature reviewed provides concrete examples of AI potential in specific applications, but few scholars have addressed a systems approach to using and regulating AI in post-secondary education and research. Establishing a robust data and AI governance framework is a cornerstone for effectively deploying these technologies in education and the peer review process. Highlighting the importance of this framework invites a future exploration and critical analysis of best practices and challenges, ensuring that data and AI are utilized ethically and efficiently within these domains. Such a governance framework should facilitate interactions among various multidisciplinary domains, thereby driving a wide range of actions essential for the ethical and effective use of AI. This collaborative and integrative approach is depicted in Figure 1, illustrating how diverse expertise contributes to the holistic application of AI in education and the peer-review process.
Figure 1: Integrative Framework for Data and AI Governance Across Multidisciplinary Domains and Actions in Education and Peer Review
Conclusion and Future Works
The Data Era, propelled by the convergence of big data and AI, presents an imminent opportunity to revolutionize education and research publishing. While this new frontier is replete with possibilities, it is also fraught with intricate challenges that necessitate innovative solutions. To unlock this era’s full potential for society’s betterment, a holistic approach must be adopted encompassing technological advancements, cooperative endeavours and stringent ethical frameworks.
In light of these developments, the potential for a more central national organization with a remit for advancing quality of education at universities and colleges is proposed to ensure a coordinated approach and to foster development of national standards and track its evolution and refinements over time. For example, the Council of Ontario Universities and similar organizations in other provinces embrace the challenging task of providing guidance and lead the development in those areas that may prove to be ethically challenging. Immediate action could be the establishment of a task force, drawing upon the expertise of existing centres and institutes at recognized universities in data science and AI developments. The primary goal is to improve the effectiveness and efficiency in the adoption process of AI in higher education.
Further research to explore the prospective advantages of data and AI for the education sector and the peer review process is required. Data and AI must be implemented in the higher education sector with a high degree of governance, in full cognizance of their ethical and practical implications for educational and peer review systems. The transformational potential of data and AI in education and research publishing can only be fully realized through a concerted and well-coordinated effort. Governance will need to know the data sources, the frequency and occurrences of data mining updates, as well as the inclusion and exclusion criteria applied. Transparency is foundational to data and AI governance.
