Do audience response systems truly enhance learning and motivation in higher education? A systematic review

Technology in higher education

In 2010, the first iPad was released in the United States, marking the beginning of a decade characterized by the introduction of tablets into the educational field. Steve Jobs was convinced that technology would transform education and change the learning process, making it more dynamic and interactive (Isaacson, 2011). Bill Gates, with some nuances, agreed with this idea: global education would be substantially improved through access to educational software platforms and high-quality, low-cost content and resources (Gates, 2010). In higher education, it is now impossible to imagine a classroom without some type of technological tool or equipment. Universities are destined to use technologies that facilitate the teaching-learning process. However, they must take responsibility for developing evidence-based education where digital tools truly foster meaningful learning and deep understanding, ensuring that their introduction into classrooms is not driven by trends or fashionable ideas.

More than a decade ago, some authors introduced the concept of Learning and Knowledge Technologies (LKT) to emphasize the educational aspect that technological tools can provide, both for students and teachers, particularly focusing on the learning methodology (Granados-Romero et al., 2014). For Lozano (2011), ICT (Information and Communication Technologies) become LKT when the potential of technology is endowed with a clear educational meaning, thus achieving more effective learning. It is, therefore, about “knowing and exploring the possible educational uses that ICT can have for learning and teaching” (p. 46). Some of these uses refer to accessing any information at any time, communicating with other students and/or personalizing education (Márques, 2012).

The impact that LKT has on education and learning has been enhanced by what is known as mobile learning (M-learning or ML). Mobile learning refers to learning that occurs through the mediation of mobile devices (Aznar-Díaz et al., 2019), taking into account its two main characteristics: first, ubiquity, which means that any content can be accessed anywhere and at any time, and second, multiplicity, which refers to the vast amount of digital resources and educational applications that students can use from their own devices (Romero-Rodríguez et al., 2021).

A historic event that further strengthened the relationship between education and technology was the lockdown that populations experienced due to the COVID-19 pandemic. This event had a significant impact not only on school education but also on higher education, as it forced a shift from in-person teaching in classrooms to a sudden virtual environment (Taboada, 2023). Universities had to redesign, in record time, the schedules, spaces, and learning processes (Bhagat, Kim (2020)), making a great effort to offer students a hybrid teaching system that allowed them to continue their academic activities during this extraordinary situation.

In the Community of Madrid (Spain), a study on best teaching practices was conducted across 230 university degree programs, interviewing 350 teachers and 260 students from all Madrid universities during the 2020/21 academic year (Fundación para el conocimiento madri+d (2021)). The conclusions are grouped into three areas: (1) Learning strategies; an increased use of active and collaborative methodologies, such as flipped classrooms, was observed, which fostered student participation. (2) Digital competencies; there was consensus on the need to continue training teachers to improve their ability to apply interactive methodologies and conduct online assessments. (3) Assessment methods; the importance of formative assessment was emphasized, focusing on the acquisition of competencies and the use of emerging digital tools.

In this context, a specific family of mobile-based tools has gained ground in university classrooms: Audience Response Systems (ARS). ARS enable real-time interaction and large-scale response collection, and they are now widely applied across disciplines. This review focuses on ARS as a verifiable instantiation of technology-enhanced pedagogy in higher education.

Audience Response Systems (ARS)

These systems are defined as applications or programs that allow the teacher to receive real-time, immediate feedback in the classroom on a posed question (Loukia & Weinstein, 2024). Such digital tools can be used to gather opinions, assess prior knowledge, or evaluate students’ learning level in real-time by utilizing an application easily accessible from a computer or mobile device. Students respond anonymously and synchronously, either individually or in groups, and the answers are projected onto the classroom screen for everyone to see. In contemporary higher education, audience response systems are commonly used to enhance student participation and engagement; widely used digital applications that function as ARS include Kahoot!, Socrative, Mentimeter, and Wooclap (Wang & Tahir, 2020; Pichardo et al., 2021; Moreno-Medina et al., 2023).

Although in this paper we refer to these applications as ARS, the scientific literature includes various terms to describe them, such as student response systems, audience response systems, personal response systems, classroom response systems, electronic feedback systems, immediate response systems, classroom communication systems, and classroom performance systems. However, instead of converging toward a unified concept, Jurado-Castro et al. (2023), in their recent systematic review and meta-analysis on this topic, coined a new term for ARS: Real-time Classroom Interactive Competition (RCIC). These authors directly connect their proposal of “real-time classroom interactive competition” with game-based learning and mobile learning.

Several decades ago, a system for anonymous and remote interaction known as the clicker became popular (Richardson et al., (2015)), and it was quickly incorporated into educational contexts to support the teaching-learning process (Caldwell, 2007). This device sparked interest in the teaching community, leading to various systematic reviews on the use of clickers in the classroom (MacArthur & Jones, 2008; Kay & LeSage, 2009; Keough, 2012). For instance, Fies and Marshall (2006) reviewed publications to determine whether learning was more effective in traditional teaching environments or those using ARS in the classroom. Although they found that the latter provided more advantages for learning, only two of the reviewed studies included a control group, and most of them collected data through satisfaction surveys without specifying other variables to analyze. They concluded that “it is impossible to assess the effectiveness of the technology itself” (p. 106).

Similarly, Hunsu et al. (2016), in their meta-analysis, examined 111 effect sizes from 53 studies involving more than 26,000 participants, concluding that “only a tiny effect of the clicker was observed on cognitive learning outcomes” (p. 116). They found that only 20% of the analyzed studies conducted pre-test measures, and only 10% of the selected studies randomized their sample.

Among the most well-known current digital applications that can be used as ARS are Kahoot!, Socrative, Mentimeter, and Wooclap. Socrative, which appeared in 2010, allows teachers to load an online questionnaire and then use it with students (see Table 1 for further details). As students respond to the questions, the system provides explanations about the questions. Fraile et al. (2021) studied how Socrative can be a valuable tool for formative assessment while also promoting self-regulated learning. Other research focused on Socrative shows apparent positive results in some psychoeducational variables, though the findings are not conclusive or generalizable (Llamas & De la Viuda, 2022).

Mentimeter was launched in 2014, and very few studies have explored its potential benefits in higher education learning. Scopus only yields around 50 articles between 2013 and June 2024 that include the term Mentimeter in their title, abstract, or keywords. One of the most recent studies (Mohin et al., 2022) concludes that it is “a powerful and flexible tool that holds the solution to improving learning and teaching in large classrooms” (p. 56). However, their research collects data from an ad hoc satisfaction survey, with responses from only 25 volunteers out of the 262 students who participated in the study. Similarly, Pichardo et al. (2021) consider Mentimeter an optimal resource for both online and in-person teaching, promoting student attention and participation, although these authors do not offer any statistical or qualitative analysis of the collected data.

Undoubtedly, the ARS that has generated the most studies is Kahoot!. It is the only application that has been the subject of three systematic reviews or meta-analyses in the last five years (Zhang & Yu, 2021; Donkin & Rasmussen, 2021; Wang & Tahir, 2020). Despite Kahoot!‘s widespread use in education, Jurado-Castro et al. (2023), in their systematic review and meta-analysis of 23 studies on RCIC, express that more research is needed to study the effect of ARS on learning, as there is no consensus on their benefits and “specific and objective reviews” are still needed (p. 3).

Finally, Wooclap is the ARS with the least history, as it entered the digital market in 2016. Wooclap has an evident educational focus and clear potential in the teaching and learning field. This is evidenced by the amount of content and resources on its website related to neuroeducation, meaningful learning, active methodologies, and peer learning. Some authors highlight the effectiveness of Wooclap in improving learning, comprehension, and participation in comparison with traditional classes (Grzych & Schraen-Maschke, 2019), as well as its influence on enhancing the performance and motivation of undergraduate students (Moreno-Medina et al., 2023). However, these studies only use satisfaction surveys to collect data and lack control groups.

Considering the empirical background described above, a brief theoretical framing can help explain how audience response systems (ARS) affect motivation and learning. Self-Determination Theory (SDT) posits that satisfaction of autonomy, competence, and relatedness underpins high-quality motivation; common ARS affordances—autonomous participation, immediate performance feedback, and structured peer interaction—can support these needs and thereby foster participation and persistence (Deci & Ryan, 2000). Cognitive Load Theory (CLT) explains how task and interface design affect learning: signaling, segmentation, and timely feedback can reduce extraneous load, whereas poorly integrated on-screen elements may split attention and hinder schema construction (Sweller, 1988). This theoretical framing guides the interpretation of the existing evidence and the selection of the psychoeducational variables examined in this review.

Objective and research questions

In conclusion, a rigorous evaluation of the potential benefits of using ARS in higher education remains necessary. The present review aims to analyze empirical studies that meet clear criteria for methodological quality and focus on this objective. Consistent with this, Table 2 synthesizes findings from prior systematic reviews on the effectiveness of ARS in learning and underscores the current evidence gap: more studies and more specific analyses are required to determine the actual impact of these applications on learning. Accordingly, this review extends prior syntheses by (i) applying PRISMA-aligned methods and predefined quality thresholds; (ii) analyzing only concrete ARS applications within a unified framework (Kahoot!, Socrative, Mentimeter, Wooclap); and (iii) focusing on the methodological quality of the included studies.

It is therefore important to know what kind of methodology has been used in ARS research, how learning variables are analyzed, and whether a real effect can be observed on these variables. In this review, the concept of “learning improvement” refers to measurable positive changes in variables such as academic performance, motivation, student engagement, or classroom participation, as reported in pre-post comparisons or between-group differences (experimental vs. control). This leads to a literature review with the general objective of verifying the level of scientific evidence on the impact of ARS use on student learning and motivation in higher education. This objective is operationalized into the following research questions:

• RQ1: What is the methodological quality of research designs on ARS?

• RQ2: What are the psychoeducational variables investigated when using ARS?

• RQ3: What effect does the use of audience response systems (ARS) have on student learning outcomes in higher education, regardless of discipline or class size?