Abstract
The COVID-19 pandemic caused major disruptions worldwide to teaching and learning activities across the education sector. We investigated the impact of COVID-19 on student engagement and performance in a third-year undergraduate science course by comparing student activity during emergency online delivery and traditional mode delivery. We found that the rapid transition to fully online teaching without any physical face-to-face teaching caused by COVID-19 resulted in learning resources being accessed at a slower rate throughout the semester. Student engagement decreased as evidenced by lower attendance at tutorials, despite this being the only virtual face-to-face activity available to students in this course. Thus, despite the smooth transition to fully online mode, it was not sufficient to prevent a decrease in student activity and participation in the online environment, and we observed a downward spiral in student engagement and motivation. Results indicate the importance of providing structure and teacher-student-peer interaction in online delivery modes.
1. Introduction
The COVID-19 pandemic represents the single largest disruption to learning in the history of humankind. It is estimated that 1.5 billion students in 200 countries were affected in 2020 by school and university closures [1]. In response to the pandemic, many higher education institutes closed their campuses and transitioned to online learning. Whilst the higher education sector in developed countries was able to ramp up preexisting online learning modes relatively quickly, there are still complex problems to address with regard to “harmonizing semesters and academic calendars as some programmes have been successfully implemented online, while others could not” [2]. Australia’s 43 universities closed their campuses and transitioned all teaching and learning online in March 2020, with this being within the first few weeks of Semester 1 for the southern hemisphere. Whilst the campuses were closed, teaching, learning, and assessment continued, and faculty and students were forced to rapidly adapt [3]. In Semester 2 of 2020 in Australia, courses were offered in a blended mode, to enable students in programs with compulsory hands-on, practical, clinical, and other essential face-to-face instruction to progress.
Online learning is a method of distance education, whereby students do not attend the university campus, but rather, learning materials and assessment are provided to them remotely. This is contrasted to traditional delivery whereby students are on campus and attend classes face-to-face with academics and their peers. Prior to personal computers and the internet, distance education occurred by correspondence using postal services. Advances in technology modernised distance education, enabling courses to be offered “virtually” for the first time in the late 1980s-early 1990s. Online delivery is popular and has become integral in the higher education setting. In Australia, for example, externally enrolled students accounted for 15% of all university enrolments in 2019; furthermore, external enrolments have increased 87% between 2001 and 2019 [4]. Online learning is delivered via online platforms known as learning management systems (LMS) or virtual learning environments (VLEs). For the student, the benefits of online delivery are greater flexibility and convenience of access to learning materials and a more tailored learning experience [5]. For faculty and higher education institutions, online learning can be cheaper and more efficient (although not always) [5]. Online delivery has its challenges, and successful teaching and learning online is dependent on many factors including correctly chosen and integrated technology, user-friendly interfaces, rapport building with students to maintain good relationships between academics and students, institutional support and training for academic staff, content development, student motivation, and self-discipline and engagement [6–9].
The interruption to teaching and learning caused by COVID-19 resulted in a rapid, unexpected transition of courses and programs online, termed “emergency online” delivery. This style of delivery is to be distinguished from the more developed methods of online delivery [10]. Research focusing specifically on the various dimensions of the interruption caused by COVID-19 in higher education demonstrates that the impact on students’ learning was largely negative. Aguilera-Hermida [11] demonstrated that students preferred face-to-face learning and subsequently struggled to adapt to emergency online delivery and that the transition to emergency online decreased students’ motivation. Similarly, Pasion et al. [12] showed that student involvement, measured as the dedication dimension of engagement, during COVID-19 decreased compared to the previous year. Similarly, Daniels et al. [13] showed students’ achievement goals and engagement significantly decreased as a result of COVID-19 disruptions.
Student engagement is a fundamental factor in learning. Student engagement refers to the “extent to which students are engaging in activities that higher education research has shown to be linked with high-quality learning outcomes” [14]. The factors influencing a student’s engagement are complex and varied [15–17]. In this study, we focused on a student behavioural aspect of engagement, analysing the frequency and timing with which online learning resources were accessed by students. Focusing on quantitative usage logs, previous research demonstrates that student engagement with online resources is lower than anticipated [18–21]. The implications of online engagement with student performance are inconclusive, with some studies demonstrating positive correlations of performance with frequency of online access [22–24] and others demonstrating nil or limited correlation [19, 25]. Whilst frequency of activity is one important variable to consider in student engagement online, timing of access and how online learning resources are used have similarly been related to student performance [26, 27].
Student engagement and motivation are intrinsically linked. Motivation refers to “the (cognitive) process of instigating and sustaining goal-directed behaviour” [28]. Whilst motivation involves unobservable and psychological factors, engagement is considered an observable behaviour [29, 30]. Schunk [28] explains that motivation cannot be observed directly but is inferred from behaviours such as choice of task and effort. There is broad agreement that motivation is a prerequisite for engagement [31]. In this study, analysing usage logs of various learning activities, we observed student behaviour that may be indicative of underlying motivation. It has been shown that motivational regulation strategies are significantly related to learning engagement [32].
Here, we report on the impact of the interruption caused by COVID-19 on student engagement and performance in a third-year science course that was intended to be delivered in blended mode but transitioned to emergency online mode. We focused on a student behavioural aspect of engagement by recording attendance at both face-to-face and online tutorials and analysing the frequency and timing of access to learning resources available via the LMS, Blackboard, throughout the semester. Furthermore, we compared student behaviour in the course delivered as an emergency online mode during COVID-19 to student behaviour in a previous year when the same course was delivered in a traditional mode. Additionally, we compared observed behaviour online to feedback provided by the students on their experience with the emergency online mode course. Within this context, the following research questions were examined: (a) when and how often were online resources downloaded by students? (b) Did online behaviours differ between modes of delivery (traditional versus blended turned emergency online)? (c) Was student overall performance a function of online engagement and/or attendance? By thoroughly examining the select observable behaviours of students, our aim is to capture what exactly students are doing, with this being a critical factor in understanding what teaching and learning materials, and what platforms, are helpful to students.
2. Context and Methods
2.1. The Course
The study was conducted at The University of Queensland (Australia) using a third-year-level undergraduate course in the discipline of natural resource science. This course is highly specialised in the subject discipline. Here, we compare the same course taught in the traditional mode in 2017 and in the emergency online mode in 2020. Course content, teaching staff, and assessment remain identical in 2017 and 2020.
In 2017, the course was offered to students in the traditional mode. The course consisted of a 3 h face-to-face didactic lecture per week and a 3 h laboratory practical per week. The weekly face-to-face lectures were recorded live using Echo360. Students had the option to attend the face-to-face lecture or watch the recording later (i.e., attendance at lectures was not compulsory). In 2017, the course had 46 students enrolled of which 23 consented to participate.
In 2020, the course was redesigned from traditional delivery with the intention of blended mode delivery. Each of the individual face-to face lectures was replaced with asynchronous video recordings (approximately 20 min) in which the students could see both the slides plus the speaker. A fortnightly, voluntary, 2 h face-to-face tutorial was introduced (which was also recorded live and could be viewed later). The weekly 3 h laboratory practical session was retained. The teaching staff and the course content were identical to that of 2017. The COVID-19 pandemic, however, forced all learning online from Week 4 onwards in 2020. Therefore, after Week 3 in 2020, the fortnightly tutorial transitioned online using Zoom and the weekly laboratory practical sessions were abandoned. In 2020, the course had 38 students enrolled of which 28 consented to participate.
2.2. Course Material
Learning resources were made available to all enrolled students via the LMS, Blackboard. Being highly specialised in the subject discipline, students are not provided with a textbook or alternative learning resources, as there is nothing suitable that covers all topics taught in this course; thus, the online resources are fundamental to course comprehension. The course was taught over a 13-week semester. Immediately following the 13 weeks of teaching was a revision week, followed by an exam week.
In 2017, the three types of learning resources available for the course on the LMS were lecture recordings, lecture slides, and modules. Modules are thorough, referenced writings on the course content, including figures, diagrams, and further reading, divided into course topics (combining groups of lectures on a similar theme). Lecture slides are the Microsoft PowerPoint slides which accompany the corresponding lecture, uploaded to Blackboard in PDF format. Both the modules and lecture slides were available to students for the entire duration of the semester. The lecture recordings comprise the PowerPoint slides, with a voice-over of the lecturer recorded live during the face-to-face lecture using Echo360 (Lectopia). Lecture recordings were automatically uploaded to Blackboard within 24 h and remained available to students for the remainder of the semester. Although in 2017 the course had 3 h of lectures per week, there were the following exceptions: there were no learning resources for Week 8 as this was a teaching-free week for all third-year students within the Faculty of Science, and there was no accompanying module available for the topic covered in Weeks 6 and 7. Week 13 was a revision week only. Assessment consisted of four written reports and an end-of-semester examination.
In 2020, the learning resources were again available to students on Blackboard. As previously mentioned, however, the live lectures (and the associated recordings) were replaced with asynchronous 20 min video recordings. These videos had been prerecorded prior to commencement of the semester, and all video recordings were available for the entire semester. Lecture slides and modules were as described for 2017, with the addition of a fifth module which covered the topic missing in 2017. The COVID-19 pandemic forced all learning online from Week 4 onwards in 2020. Hence, the laboratory practicals were conducted face-to-face for the first three weeks but were then replaced with videos that were uploaded to Blackboard whilst the face-to-face tutorials were replaced with synchronous online tutorials conducted via Zoom. These online tutorials were recorded and subsequently made available to students via Blackboard within 24 h. Students could therefore attend the synchronous tutorial session (Weeks 5-13) or watch the recording later. Tutorials in Weeks 1-3 were face-to-face in-classroom instruction but were also recorded. The interruption due to COVID-19 caused minor university-wide modifications to the overall structure of the teaching weeks of Semester 1 in 2020. Specifically, there was a pause between Week 3 and Week 4 where no teaching took place to facilitate a transition to online-only learning, and after Week 13, two additional teaching weeks (ATWs) were added for courses that required additional teaching time to balance interruptions cause by COVID-19. For the course being examined in this study, however, no additional teaching material was introduced in the ATWs, and hence, the ATWs were equivalent to the revision week in 2017. Assessment consisted of four written reports and an end-of-semester examination.
2.3. Data Collection, Analysis, and Limitations
The study reports data only from those students that provided written consented to participate in the study. Ethics approval (number 2019000698) was granted from The University of Queensland Low and Negligible Risk Ethics Sub-Committee. Blackboard click data was used to track student access to all learning resources throughout the semester. A count, along with student ID and a timestamp, is made each time a student clicks on a file link within Blackboard. Click data was analysed with regard to total frequency of access and timing of access. There can be no certainty about what the student does with the file after clicking, which is a limitation of this data. For further discussion about click data limitations, refer to McKenna and Kopittke [33].
Tutorial attendance at both the face-to-face sessions (Weeks 1-3) and the synchronous Zoom session (Week 4 onwards) was recorded by student ID and examined across the semester. Physical and online attendance was compared directly to downloads of online lecture or tutorial files. Overall performance was collated at the conclusion of the semester after grades had been finalised. Overall performance combined student results on all forms of course assessment. Performance was examined as a function of both attendance and online engagement. Performance was compared with attendance and total downloads of learning resources using IBM SPSS version 27 (IBM Corporation, New York) with linear regressions. Analyses comparing the frequency of access to different types of learning resources (recordings, slide, and notes) between online delivery and traditional mode delivery were performed using the independent-sample test using IBM SPSS version 27 (IBM Corporation, New York).
2.4. Student-Led Observation for Course Improvement (SLOCI)
In addition to the click data from the LMS, in 2020, we collated student feedback on the emergency online mode. Feedback was provided by students under a program referred to as student-led observation for course improvement (SLOCI), which is administered by the Faculty of Science at The University of Queensland and is freely available to all courses at the request of the course coordinator. Feedback was collected from participating students in focus groups led by a student peer three times throughout the semester in 2020: Week 2, Week 6, and Week 12. The questions asked of students were formulated by the course coordinator and the SLOCI team.
3. Results
Student engagement was examined using lecture (2017) or tutorial (2020) attendance in addition to student activity online (both lecture and tutorial attendance was voluntary). Tutorial attendance was lower than attendance at traditional face-to-face lectures (Figure 1). Average attendance at tutorials in the online mode was 35%, whereas average attendance at face-to-face lectures in the traditional mode was 59%. A downward trend in attendance as the semester progressed was evident in both modes, although it was more pronounced toward the end of the semester in the online mode (compare Figures 1(a) and 1(b)). From Week 4 onwards in the online mode, tutorials moved online in a synchronous format due to the COVID-19 interruption.
(a)
(b)
We examined when learning resources were being accessed as a function of teaching week. This analysis used the total number of downloads of each resource, and thus, any duplicate downloads by unique individuals were incorporated. There was a clear pattern of increased engagement online as the semester progressed into revision and examination weeks in both modes (Figure 2). The last three weeks of the semester accounted for 43% of all downloads in the traditional mode (Figure 2(a)) and 46% of all downloads in the online mode (Figure 2(b)). The data also demonstrated a difference in resource preference between the two modes. Students in the online mode downloaded the lecture recordings and modules more frequently than did students in the traditional mode, whereas students in the traditional mode downloaded lecture slides more frequently than did students in the online mode. In both modes, students downloaded lecture slides more frequently in the last few weeks of the semester. In the online mode, it is noteworthy that total downloads of resources in Week 4 (i.e., the first fully online week due to the COVID-19 interruption) were similar to those in the first three weeks of the semester; from Week 5 onwards, total downloads of online materials were greatly reduced until the end of the semester.
(a)
(b)
To further examine the usage of online learning resources, we analysed the cumulative number of resources downloaded (as a proportion of total available) across the semester. We examined the number of each type of resource downloaded as a proportion of total available resources each week. Duplicate downloads by unique individuals were excluded on a weekly basis (i.e., where a student had downloaded an individual file multiple times in one week, only one download was counted). Also, this analysis considers the difference in numbers of students enrolled in the two years by reporting on a percentage basis. This analysis demonstrated both when resources were viewed initially during semester and the resource preferences. In the traditional mode, the cumulative downloads of resources were gradual over the semester, with students downloading resources at a steady rate (Figure 3(a)). In the online mode, even though all of the learning resources were made available at the commencement of the semester (cf. traditional mode where the lecture recordings were only released progressively during the semester), downloads of resources remained lower until Week 13 when there was a marked increase in downloads of lecture recordings, modules, and lecture slides (Figure 3(b)). For example, 50% of available modules had been downloaded by Week 6 by students in the traditional mode, whereas 50% of modules had been downloaded by students in the online mode by Week 9. Comparing the patterns of download activity between Figures 2 and 3, it is evident that although there was a pronounced increase in downloads at the end of the semester for both the traditional and online modes (Figure 2), for the traditional mode, this increased activity largely represents students that are revising material that they had already accessed earlier in the semester, whilst for the online mode, this increased activity at the end of the semester represents students that are largely downloading materials for the first time (Figure 3). Students in the traditional mode preferred lecture slides—81% of available slide files were downloaded by the end of the semester, which was significantly higher than for modules at 70%, which was similarly significantly higher than for recordings at 62% (, Table 1). In the online mode, students preferred modules—86% of available module files were downloaded by the end of the semester which was similar to lecture recordings at 78% but significantly higher than for lecture slides at 40% (, Table 1). Tutorial recordings were the least preferred resource in the online mode with only 17% of files downloaded.
(a)
(b)
Given the low level of tutorial attendance in the online mode, we examined if students were utilising the tutorial recordings to substitute a missed synchronous tutorial session. Overall, 40% of the students absent from the synchronous tutorial session downloaded the recording, and 20% of students supplemented their attendance at the live session by also downloading the recording (Figure 4(b)). The tutorial sessions in Week 1 and Week 3 were run as face-to-face sessions where attendance was relatively high (Figure 1); tutorial sessions from Week 5 onwards were run synchronously via Zoom. In the traditional mode, a significantly greater proportion of students utilised lecture recordings; 85% of the absent students substituted the missed face-to-face lecture with the recording and 62% of the present students supplemented their attendance by downloading the recording (Figure 4(b), , Table 1). However, significantly more students in the online mode accessed lecture recordings (78%) than did those in the traditional mode (62%) (, Table 1).
(a)
(b)
After examining student attendance and patterns of engagement with online resources, we compared these student behaviours with student performance. First, we examined the relationship between attendance and overall performance. In the traditional mode, students that attended face-to-face lectures most frequently resulted in slightly lower levels of achievement than students who attended less frequently (Figure 5(a)). In the online mode, there was a slight positive relationship with attendance at tutorial sessions and performance (Figure 5(b)), with students attending 50-100% of tutorials performing better than students attending 0-50% of tutorials, but the relationship was not significant. Average overall performance was 73% in the online mode and 70% in the traditional mode.
(a)
(b)
Next, we compared the number of resources viewed by individual students (as a proportion of total available resources, excluding duplicate downloads), with overall course performance. In both modes, there was a small but positive trend with resources downloaded and performance. Students that downloaded a greater proportion of online recourses attained higher achievement (Figure 6); however, the trend was not significant.
(a)
(b)
Finally, we considered the student feedback on the online mode course collected throughout the semester in 2020. Student feedback received as part of the SLOCI initiative focused largely on the impacts of COVID-19 on the course itself and the impact on student engagement and motivation. Per the initial feedback received in Week 2, prior to COVID-19 interrupting learning, the students looked forward to the flexibility offered by a blending learning. However, they were also astute in predicting the difficulties associated with self-driven learning and the possibility of not being able to keep up with learning materials throughout the semester. Feedback received in Week 6, two weeks after COVID-19 forced all learning online, focused largely on the effects of COVID-19 on the course delivery. Two broad themes were identified by the students. First, students unanimously identified that this course, having been intended to be delivered in blended mode, was well-placed to be transitioned fully online. They stated that there was a “notable discrepancy” in the quality of the learning resources prepared for this course compared to other courses; another student remarked that this course “has obviously had the upper hand on a lot of courses that weren’t prepared to cope with the transition.” Furthermore, students recognised that teaching staff for this course were experienced and skilled in online delivery, whereas teaching staff from other courses were experiencing technical difficulties. However, a second theme identified by the students was that the move to fully online because of COVID-19 had a negative impact on student engagement and motivation and diminished the learning experience. Students were disappointed to miss out on face-to-face contact, with both peers and the teaching staff, in the course. One student stated, “I find a lot of value in interacting with the lecturers and classmates in person, especially if you are struggling with something and you can’t really explain it in words.” Students commented that the absence of face-to-face contact would diminish the learning experience, particularly with regard to the laboratory practical component, with students noting, “You retain so much more when you are actually doing it” and “you learn from the experience of doing it.” Motivation and engagement levels dropped with the move online, the lack of face-to-face contact, and the less structured format the course took. We observed both in the online patterns of behaviour and in the feedback received from students a downward spiral effect; students lost motivation due to the lack of regularly structured face-to-face activities, which resulted in low levels of engagement in learning resources, which meant they did not attend the tutorials, because they were “not on top of everything” despite it being the only scheduled virtual face-to-face contact. Students suggested more structured activities such as quizzes to provide an external motivator and to maintain engagement to keep up-to-date with learning materials as well as to gauge learning progress.
4. Discussion
4.1. The COVID-19-Induced Move to Fully Online Teaching Reduced Student Engagement and Attendance
Attendance was greater at face-to-face lectures in the traditional mode than it was at tutorials in the online mode (Figure 1). This was an interesting observation because in the traditional mode, students had a choice to either attend the face-to-face lecture or watch the recording of the lecture later (i.e., lecture attendance was not compulsory); additionally, these students were also attending weekly face-to-face practical laboratory classes. However, in the online mode, the tutorials, although voluntary, were the only scheduled time students could meet with their peers and the lecturer to discuss lecture material. Furthermore, from Week 4 onwards, due to COVID-19, the synchronous Zoom tutorial became the only virtual face-to-face contact students had with the lecturer. Yet, despite it being the only scheduled contact and despite students overwhelmingly stating that they greatly missed the face-to-face interactions, attendance rates at the Zoom tutorials were markedly lower than attendance at the face-to-face session in Week 1 and Week 3 (Figure 1). Furthermore, although the Zoom tutorials were recorded and posted on Blackboard, only 40% of the absent students substituted their attendance at a live session by watching the recording (Figure 4). We consider it likely that this decreased attendance at the online synchronous sessions from Week 5 was representative of the negative impact on COVID-19 on student engagement. As part of SLOCI, students commented that they did not feel comfortable attending tutorial sessions because they recognised that they were not up-to-date with learning materials and therefore had no questions to raise at the tutorial. The decreased engagement with learning materials resulted in decreased motivation to attend tutorials, which demonstrates the significant interrelatedness between engagement and motivation [28, 31, 32]. The student feedback explains our observed pattern of behaviour which demonstrated that tutorials and the tutorial recordings were the least preferred resource in the blended mode (Figures 1, 3, and 4). The low levels of engagement observed may also be attributed to the type of synchronous online activity (i.e., a tutorial) offered. For example, in medical education, flipped classrooms, online practice questions, and teleconferencing in place of face-to-face lectures have been suggested as a way of maintaining learning during COVID-19 restrictions [34].
4.2. Decreased Engagement Was Evidenced through Decreased Downloads during the Semester
It was clear that the number of the learning resources downloaded progressively across the semester was lower during the COVID-19-interrupted online mode than during the traditional mode. Comparing the patterns of downloads as a function of total numbers of downloads with the cumulative downloads each week (Figures 2 and 3), it is evident that whilst there is a pronounced increase in frequency of resources being downloaded at the end of the semester in both modes (Figure 2), students in the traditional mode are revisiting material presumably for revision purposes, whereas the increase in activity in the online mode represents a large proportion of students downloading resources for the first time in an effort to catch up (Figure 3). We consider this to be reflective of the greater structure and direction afforded by the traditional mode compared to the online mode. In the traditional mode, weekly face-to-face lectures provide students with a guided rate of progress through learning resources, with the lecturer supporting that guidance directly or indirectly in a lecture. Furthermore, students in the traditional mode have both the choice and the extrinsic motivator of attending a face-to-face lecture that enables social interaction with peers and lecturers. Student feedback in SLOCI was reflective of the observed trends in download rates. Students commented that they would like to have more structured activities throughout the semester to help guide their learning and to provide motivation to progress through learning resources more consistently. In the feedback, students suggested periodic quizzes be incorporated. Periodic quizzes could provide extrinsic motivation and encouragement to keep up with learning resources whilst also providing feedback to students about content not yet mastered [35–37].
It was also evident when examining the frequency of downloads (Figure 2) that student online engagement was directly affected by COVID-19. Download frequency in Week 4 of the online mode, being the first week of online learning, was comparable to that in Weeks 1-3. Download frequency was however greatly reduced in Week 5 through to Week 12 compared to both earlier in the semester in the online mode and also compared to the download frequency in the traditional mode at the same period in the semester (Figures 2(a) and 2(b)). The transition to online learning due to COVID-19 negatively impacted student engagement with learning resources. We consider this change in behaviour, with regard to download patterns, to be indicative of reduced levels of motivation because motivation drives engagement in learning. Similarly, a significant decrease in achievement goals and engagement because of emergency online delivery due to COVID-19 was demonstrated in Daniels et al. [13]. Aguilera-Hermida [11] concluded that “when students are not motivated their cognitive engagement is lower and vice-versa.”
Our observations that student engagement with the learning resources was low during the COVID-19-induced move to fully online teaching were interesting. Student feedback, as part of SLOCI, informed us that, compared to other courses that they were undertaking simultaneously, the quality of the learning resources provided in the present course was very high. We conclude that this was primarily because our online learning resources had all been deliberately prepared prior to the commencement of the semester as part of the transition to blended learning. Indeed, studies have demonstrated the importance of pedagogical preparedness in the success of transitioning learning online amidst the COVID-19 pandemic [38, 39]. Whilst students were able to recognise the quality of learning resources, the combination of reduced engagement and motivational factors induced by COVID-19 resulted in the learning resources being underutilised in emergency online delivery. Dunn and Kennedy [40] reported that underutilisation of online resources is common and identified that the type of online learning environment implemented can influence student attainment and that there is a relationship between online usage and student engagement and motivation. Furthermore, student attitude toward online learning has been shown to have a negative impact on motivation and engagement in online modes of learning [41].
4.3. Learning Resource Preferences and Download Patterns
Although the transition to fully online teaching because of COVID-19 resulted in decreased engagement compared to a traditional mode, the motivational patterns of the students was similar—the pending examination resulted in a profound increase in student engagement with the resources. For both modes, the greatest number of downloads occurred in the final few weeks of the semester (Figure 2). This pattern, observed here in both a traditional mode and a COVID-19-induced online mode, is similar to that previously reported where engagement with learning resources increases near assessment deadlines [21, 42].
There were however differences in resource preferences between the modes. Module files were the most frequently downloaded resources by students in the online mode whereas students in the traditional mode had a preference for lecture slide files (Figures 2 and 3). Students in the online mode downloaded the lecture recordings significantly more frequently and more consistently than did the students in the traditional mode (Figure 3 and Table 1). We feel this is reflective of the different file types in addition to the effect of the delivery mode. The lecture recording files in the online mode were 20 min recordings including slides plus speaker view; further, in the online mode, these recordings were the only lecture material available to students. In the traditional mode, lecture recordings were 50 min Echo360 recordings of the live lecture incorporating the lecture slides. Students in the traditional mode had a choice to attend the face-to-face lecture or watch the recording later. Therefore, we see a greater uptake in the lecture recordings by students in the online mode compared to students in the traditional mode. Rates of attendance demonstrated that students in the traditional mode prefer to attend the face-to-face lecture and are therefore less reliant on lecture recordings. Slide files were downloaded significantly more frequently and consistently throughout the semester in the traditional mode than in the online mode (Figures 2 and 3 and Table 1). This is likely reflective of the difference in lengths of lecture recording files between the two modes and that students in the traditional mode may have downloaded the slide file to complement attendance at the lecture, either to annotate with notes during the lecture or to revise shortly after the lecture. The greater uptake of slide files in the revision and examination weeks (Figure 2) in both modes is likely reflective of the greater convenience afforded by slide files over lecture recordings (which also contain the slides). Students can focus on the written material of the slides and progress through material at their own pace, rather than committing to watching a 20 or 50 min long recording, which may be more efficient for revision purposes.
4.4. Engagement and Performance
By analysing the effect of behaviour on student performance, we found small but positive relationships in most instances. In both modes, the more online resources students downloaded, the higher their performance (Figure 6). The effect was slightly greater in the online mode than in the traditional mode. Given the interruption by COVID-19 to the blended delivery mode, by forcing all learning online at Week 4, online engagement became the only means by which students could continue their learning, and hence, it is perhaps not unexpected that it is important in determining performance. Similarly, in the online mode, students that attended the tutorials (both face-to-face and virtually) more frequently attained slightly higher achievement than students who did not attend frequently. After Week 4, synchronous zoom tutorials became the only avenue students had to interact with peers and lecturers for the remainder of the semester; the results demonstrated that students that attended gained a small advantage over students that did not attend (Figure 5(b)). Our results demonstrated that under the unexpected circumstances caused by COVID-19, synchronous online activities, such as tutorials via Zoom, are a valuable substitute for face-to-face in classroom instruction. Interestingly, we observed a slightly negative impact of attendance on performance in the traditional mode (Figure 5(a)). We did not observe any significant difference in overall course performance between the two modes of delivery examined.
5. Conclusion
Overall, we observed a direct impact of COVID-19 on student engagement in terms of both attendance at tutorials and engagement with online learning resources. Specifically, we observed important differences in the cumulative download rates, with the transition to online-only resulting in the students accessing the learning resources at a slower rate across most of the semester. This decrease in student engagement during the semester was also evidenced by the lower attendance at tutorials despite these being the only avenue for interaction with student peers and the lecturer in the online-only environment. We consider that the observed patterns of behaviour as a measure of engagement are indicative of student motivation. The decrease we observed in student engagement during fully online delivery was not due to the comparative quality of learning resources (the intention to deliver in blended mode resulted in a smooth transition to fully online due to COVID-19). However, even this was not sufficient to prevent a decrease in student engagement and motivation in the fully online learning environment, and we observed a downward spiral in student engagement and motivation. Results highlight the critical importance of providing structure and teacher-peer-student interaction in online learning modes.
Data Availability
The data that support the findings of this study are available from The University of Queensland, but restrictions apply to the availability of these data, which were used under ethical approval for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of The University of Queensland.
Ethical Approval
Ethics approval number 2019000698 was granted from The University of Queensland Low and Negligible Risk Ethics Sub-Committee, The University of Queensland. Students enrolled in the courses were informed of the study via a Blackboard announcement (which was also forwarded to their UQ email address) and in the first tutorial by way of a Participation Information Sheet. Ethics approval number 2019001666 was granted by The University of Queensland Low and Negligible Risk Ethics Sub-Committee for the purposes of collecting student feedback as part of the SLOCI initiative.
Consent
Only those students that provided Consent to Participate were included in this study.
Conflicts of Interest
We have no conflict of interest to declare with this manuscript. The authors had complete access to the data presented within the manuscript “Online Engagement during COVID-19: Comparing a Course Previously Delivered Traditionally with Emergency Online Delivery.”
Authors’ Contributions
Author BM was responsible for data curation, investigation and formal analysis, and writing the original draft preparation. Author PK was responsible for conceptualization and reviewing and editing. Author CH was responsible for data curation of the SLOCI data.