www.ijcrsee.com
267
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
Introduction
Teachers play many roles in classrooms, including inspiring students, stimulating their interest in
the subject, and serving as a source of knowledge and information. Besides, commitment and passion are
required, while a combination of teaching paradigms is needed to maintain momentum and transform the
classroom into a learning environment full of enthusiasm and interest (Powell et al., 2012).
Teachers increasingly complain that their students do not show interest, do not prepare for lessons,
have passive attitudes toward learning activities, and limited awareness of their own learning process
(Buijs and Admiraal, 2013). This, and the contemporary competence-oriented higher education, requires
a correspondingly strong focus on pedagogy. But there is also a growing recognition that to develop
the knowledge and skills needed for the 21
st
century, we also need innovations in the eld of teaching
(Paniagua and Istance, 2018).
As explained by Heacox and Pengal (2009), the greatest challenge for today’s teacher is to respond
to the increasingly diverse learning needs, styles, and social backgrounds of students, which affect their
willingness to learn, learning styles, motivation, interests, attitudes to learning, and self-condence. He,
therefore, suggests that differentiation of instruction, which puts students at the center of teaching and
learning and also allows learning needs, learning styles, and interests to guide instructional design, can
be very helpful, including varying the pace of learning, the level of difculty and the way instruction is
delivered (Heacox and Pengal, 2009).
The traditional lecture is not an effective learning environment for many students, as many do
not actively participate during a traditional lecture (Bajpai, Biberman and Ye, 2019). The goal is to keep
students active, and approaches to learning are changing. In student-centered learning, the focus is on
students’ needs, abilities, interests, and learning styles, and the teacher is only a facilitator of learning. Due
The Impact of the Covid-19 Pandemic on Higher Education Students’
Perceptions of Educational Applications and Platforms
Katarina Rojko
1*
, Nuša Erman
1
1
Faculty of Information Studies, Novo mesto, Slovenia,
e-mail: katarina.rojko@s.unm.si, nusa.erman@s.unm.si
Abstract: The contemporary development of education is marked by the rapid expansion of online educational applications
and platforms. Consequently, it is becoming clear that teachers must consider them and in light of their changing role, know
what they can offer in their eld to be able to recommend them to students to get additional knowledge or even to integrate
them into their teaching. As students in the eld of computer science and informatics are particularly accustomed to considering
online knowledge resources, we decided to investigate their perception of educational applications and platforms. Based on
the analysis of the results of their testing of educational platforms and applications, we found that for them the most important
is content, followed by personal preferences, reason, user experience, price, etc., and only in the last place is a certicate. The
most frequent word in their research reports proved to be knowledge, which we included under the code reason, followed by
research, content, time, free, variety, and quality. It also turned out that students’ experience of testing educational applications
and platforms is predominantly positive and has even improved over the course of the last three years, which we attribute to
the effects of the Covid-19 coronavirus epidemic. The comparison of pre-pandemic and post-pandemic data also revealed that
positive sentiment came to the front, while students now prioritize user experience, reason, and quality over the content and
personal preferences compared to the pre-pandemic period, while they are still aware of the need for exploration.
Keywords: e-education, e-learning, higher education teaching, educational applications, educational web platforms.
Original scientic paper
Received: March, 30.2023.
Revised: May, 11.2023.
Accepted: May, 24.2023.
UDC:
004.9:37.011.3-057.85
37.048:[616.98:578.834
10.23947/2334-8496-2023-11-2-267-279
© 2023 by the authors. This article is an open access article distributed under the terms and conditions of the
Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
*
Corresponding author: katarina.rojko@s.unm.si
www.ijcrsee.com
268
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
to the development of the World Wide Web and search engines, ‘on-demand’ information has become a
reality (Hirsh, 2018). In this context, students need to be active and responsible participants in the learning
process, and the teacher plays a key role in the whole process.
E-learning uses various information-communication technology (ICT) tools to make the teaching
and learning process more effective (Bajpai, Biberman and Ye, 2019). As a result, classroom lectures,
whose main objective is to cover the content of the course, can be expected to become obsolete, as can
the willingness of students to attend these classroom lectures, as technology allows them to learn at
their own pace and in their own time (Moreno, 2018), expanding the possibilities for individualisation and
personalisation of learning (Urbančič, 2021). More and more people are realising the benets of learning
at a pace that suits them best, which is why e-learning is on the rise.
E-learning allows the teacher and the student to be independent in terms of location and time, making
it a form of distance learning (Bregar, Zagmajster and Radovan, 2010). Furthermore, the advantage of
online education is that many courses are very affordable for the value they offer and are aimed at almost
every individual in the world, whereas traditional education involves a limited number of students (Krastev,
2019), which is an additional reason for the rapid rise of educational apps and platforms.
While the concept of e-learning generally refers to the methods of education implemented using
ICT, ICT in the context of e-learning represents only part of the technological solutions that allow us to
complement or enrich the basic foundations and concepts of traditionally based learning (Jagodič, 2010).
Nevertheless, ICT is a potentially powerful tool for expanding educational opportunities, both formal and
non-formal (Adenusi, Adebayo and Oni, 2019), and it is the development of ICT that has brought about
signicant changes in the eld of education (Semerci and Aydın, 2018).
However, we recognise that teachers need to have certain digital competencies to be able to use
online platforms and applications for communication, teaching, and research. Therefore, national projects
to promote the use of ICT by (higher education) teachers are very welcome. Teachers play a central role in
integrating ICT into the classroom, so improving their digital competencies and attitudes is crucial for the
effective integration of ICT in education (Semerci and Aydın, 2018). In addition, it is important to be aware
that today’s students are so-called digital natives, who are expected to have the knowledge and skills that
allow them to handle ICT tools ‘naturally’ (Boh Podgornik et al., 2016), so it is not surprising that students
often expect this from teachers as well.
The introduction of the use, integration, and dissemination of ICT has ushered in a new era in
educational methodologies, radically changing traditional teaching methods and learning patterns in the
eld and offering modern learning experiences for both teachers and students (Adenusi, Adebayo and Oni,
2019). Therefore, different teaching/learning strategies need to be applied in education. This is because
digital technologies are often used to support the extension of university education into the home, social
and work environments (Castañeda and Selwyn, 2018).
As a result, ICT has a broader role in teaching and learning activities into which it can be integrated
as a tool in the learning process (Lautenbach, 2014). Furthermore, the important role of ICT can also
be found in activities such as content creation, student performance monitoring, training, knowledge
management and organisation, and formative monitoring (Margaret et al., 2018). Black and Wiliam
(2009), two of the most referenced authors, established a conceptual framework for understanding
formative monitoring through a series of publications that can be traced back at least to the mid-1990s.
Formative monitoring can be imagined as a “bridge between learning and teaching”. It is based on ve key
strategies: clarication, participation in setting and understanding learning intentions and success criteria;
designing classroom activities that provide evidence of learning; providing feedback; activating students
to become teachers to each other; and activating students to self-manage their learning (Palir Mavrič,
2017). The integration of ICT was identied as a good practice, particularly in stimulating and increasing
students’ motivation. But the use of technology in learning does not automatically result in engagement
in learning activities and improved learning outcomes (Sinha et al., 2015), and for this reason, well-
designed computer-supported collaborative learning (CSCL) environments for collaborative learning and
problemsolving might intensify learning if used based on a teacher’s instructional support (Pietarinen,
Palonen and Vauras, 2021).
Moreover, teachers and students must understand that independent learning using online
applications and platforms has a major disadvantage if not carefully addressed and that is a lack of
collaboration. Namely, students generally do not recognise the advantages of collaborative learning, and
therefore teachers should play an important role in encouraging and inuencing student participation and
achievement (Webb et al., 2019). The teacher’s role has thus changed to act as a coach and facilitator of
students’ thinking while modelling the learning process (Anderson, 2002).
ICT tools are thus playing an increasingly important role in the renewal of educational methods.
www.ijcrsee.com
269
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
This is manifested in the form of digital learning materials instead of printed books, interactive materials,
e-communication between teachers and students, etc. (Berényi and Deutsch, 2018). While the pandemic
has increased the use of and changed teachers’ attitudes toward digital learning materials, quality teaching
in the digital age would require supporting teachers with training to make working with digital materials
routine (Mohar and Kovač, 2021). ICT tools used to create and disseminate knowledge including equipment
and software solutions have become an indispensable part of modern culture, which is spreading globally
through education (Bajpai, Biberman and Ye, 2019). Software that supports the didactic process in higher
education includes internet browsers, ofce suites, communicators, specialised software suitable for
classroom use, software that supports group work, and above all e-learning applications and platforms
(Grooms, 2018), which are the focus of our research.
As the use of educational applications and online platforms is growing rapidly (e.g., Data Bridge
Market Research, 2022 or Dhawal, 2021), we decided to explore their use for study purposes. To this end,
we analysed the opinions of students from four Informatics and Computer Science degree programmes,
two rst-degree programmes and two second-degree programmes.
We started the survey in 2019 when we rst asked students enrolled in their nal year of study on
these four study programmes about which apps and online platforms, they use to support their studies.
Based on the responses we received, we found that the use of apps and online platforms is much more
widespread among students and not limited to those suggested by teachers. In addition, the students
presented in their research reports the results of the testing of the apps and online platforms they use to
complement their studies.
As today’s teachers (and especially teachers in information technology) and computer science
programmes, due to the specicity of these programmes) are increasingly aware of the need to use
and even compete with online educational opportunities, it has become clear that they need to change
their teaching methods and see their profession as adding value and upgrading the information that
students can obtain via the Internet. Our main research objective is therefore to present the ndings of a
qualitative analysis of students’ views when testing educational applications and online platforms, which
we believe could encourage higher education teachers to make greater use of them in their teaching,
not as a substitute, but as an additional source of ‘on-demand’ knowledge and motivation for learning.
Finally, this paper aims to encourage higher education institutions to reect on how encouraging the use
of educational apps and online platforms can enable students to perform even better in exams, while also
spreading awareness that they can deepen their knowledge in this way too.
Materials and Methods
Statistics (e.g., Wise, 2022) show that e-learning has grown exponentially in recent years. Studies
also show (e.g., Suwal and Singh, 2018) that the use of e-learning applications and platforms is well
accepted by students as it gives them positive experiences, and consequently, there is a growing need
for more integration of e-learning applications and platforms into existing learning activities and practices.
In the ood of different types of educational apps and online platforms, it is difcult for a teacher to decide
which app or platform is appropriate to use and recommend to students. In addition, more and more
new apps and online platforms are emerging, making it difcult to keep up with these advances and the
increasing range. In this context, we have identied the need for a systematic review of the educational
apps and platforms that students individually choose to use, as well as the way they are perceived. To do
so, we have conducted a survey in which students themselves, based on their study interests, chose an
app or online platform to test. They presented their views gained during the testing in research reports,
which we then used to achieve the aim of our research: to show that teachers need to change their role,
as the internet offers instant information on almost any topic and as educational apps and platforms are
gaining in importance. Accordingly, teachers also need to change their teaching methods to offer students
all they can gain from e-learning.
Based on the content of the students’ research reports, we identied the codes and the most
frequently written words to give their perspective on the impact and usefulness of some educational apps
and platforms in their elds of study, as the research also aims to encourage teachers to reect on how
the integration of apps and platforms in their courses can bring added value. In addition, the Covid-19
coronavirus pandemic has further stimulated the development of online educational resources. Based on
this research objective, the following research questions (RQs) were set:
RQ1: What has proved to be most important in the testing of educational apps and platforms by
students?
www.ijcrsee.com
270
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
For this question, we were looking for codes that could be extracted from a close reading of the
assignments that students had prepared based on testing educational apps and platforms. We assumed
that students would put the user experience at the center, and quality would be the decisive factor (as in
Rojko, 2020). On the other hand, although some apps and platforms provide certicates and recognition
of the knowledge acquired, we assumed that the absence of formal recognition would demotivate students
to learn online on their own initiative.
RQ2: Which words stand out in the research reports of students who have tested educational apps
and platforms?
We asked students to use different criteria to test different applications and platforms for their
research reports, mainly qualitative but sometimes also quantitative. We anticipated that the words
quality, user experience, quantity, price and certicate and their synonyms would appear most frequently.
We were also interested in whether positive or negative experiences predominated.
RQ3: Has the impact of the Covid-19 coronavirus pandemic changed the way students view
educational apps and platforms?
We expected that some new dominant words would appear in the more recent research reports
(from the second half of the academic year 2021/22) compared to the reports from the period before the
start of the epidemic (from the rst half of the academic year 2019/20). Namely, we observed that the
Covid-19 pandemic was reected in an increased motivation of students to search for additional online
education resources and in their increased willingness to try them out, as distance education had been
the only option for some time.
The survey was launched in 2019 when we asked students enrolled in the nal year of four
Informatics and Computer Science degree programmes from Slovenia, which educational apps and
platforms they use to support their studies. In addition, the majority of these students also produced a
research seminar paper based on the results of their testing.
Students chose and tested the following educational apps and/or platforms (in alphabetical order):
ArtofProblemSolving, Astra.si, Bussuu, CK-12, Code With Mosh, CodeAcademy, Codewars, Coursera,
Cybrary, Drops, Duolingo, edX, freeCodeCamp, FreeVideoLectures, FunEasyLearn, Khan-academy,
KodeKloud, Laracast, LinkedIn learning, Mathway, Memrise, Micro:bit, Mimo, MojeZnanje.si, Photomath,
Pluralsight, Programming Hub, Qlango, Scratch, Skillshare, SoloLearn, SQLZOO, Stack Overow,
Symbolab, Treehouse, Tutorialspoint, Udemy, Udacity, W3Schools, Wolfram Alpha, Yoast and YouTube.
Note that some students chose the same apps and platforms, and Udemy was the most repeated platform,
18 times.
Some of the educational apps and platforms chosen specialise in one area, others cover several
different areas, and students were instructed to focus on one specic subject area for any comparisons.
As can be seen from the list of names, some of the listed platforms cover different areas of knowledge,
although all students focused on computer science, while the listed apps focus on computer science (most
frequently tested), mathematics (medium frequency), and foreign language learning (least frequently
tested). The majority of the apps and platforms analysed were international, but two Slovenian platforms,
Astra.si and MojeZnanje.si, were also included.
Among the 140 students enrolled in the nal year of their rst (professional study programmes)
and second (masters study programmes) cycle studies in the academic years 2019/20, 2020/21 and
2021/22, 113 (81%) submitted their research seminar paper. Among these, we further excluded those
reports in which students tested other applications that they had used during their studies but were not
relevant to our research - e.g., for storing data and literature, such as Dropbox, Evernote, Google Drive,
OneNote and Zotero. Thus, the nal 90 research reports (50 reports by rst-cycle students and 40 reports
by second-cycle students) were considered, out of these 29 from the academic year 2019/20 (rst half
of the year (in continuation referred to as the pre-pandemic period) 16 reports), 36 from 2020/21, and 25
from 2021/22 (second half of the year (in continuation referred to as post-pandemic period) 10 reports).
We considered the content of research reports as transcripts, which were then subjected to qualitative
analysis to answer our research questions. We resorted to manual coding and used the Atlas.ti qualitative
analysis tool.
Manual coding (using the open axial coding methodology) required close reading and careful
consideration of the students’ opinions and observations, taking special care to avoid concepts we had
learned from studying the literature and our views and expectations. The attribution of concepts was
done both by direct conceptualisation and synonymy. The selection was made based on the conceptual
framework and within it based on recurring terms in the students’ assignments. Individual terms were
further evaluated and linked to semantically equivalent terms, which were grouped into key categories
(codes) based on their meaning.
www.ijcrsee.com
271
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
Atlas.ti was used to search for the most frequently repeated words in the assignments. Although this
tool also allows the preparation of word clouds based on Slovene texts, it only formats them appropriately
in some of the world’s leading languages. As the tool is not able to combine Slovenian words in different
conjugations, does not correctly identify word types, etc., we rst translated the 90 analysed seminar
reports, with an average size of one author eld (The author’s pole consists of 30 000 characters (letters,
numbers, punctuation, spaces) or approximately 16 typed pages.), into English. We then extracted nouns,
verbs, adjectives, and adverbs, followed by manual removal of the remaining inappropriate words. Finally,
for the most common words, we rst considered separately the terms for the full spectrum of experience
from positive to negative, and then all the remaining most commonly used words.
In addition, we also analysed the research reports from the rst half of the 2019/20 academic
year (considered as pre-pandemic period reports) and the research reports from the second half of the
2021/22 academic year (considered as post-pandemic period reports) separately and used the results
to compare the two periods, which led us to detect the changes in the testing experience as well as in
students’ attitudes towards educational apps and platforms. To do so, we applied chi-squared test at
signicance level of 0.05 to nd potential association between pre- and post-pandemic periods regarding
the testing experience and students’ attitudes. In the case of a statistically signicant chi-squared test, the
association was conrmed and was then tested for strength using the Cramer’s V contingency coefcient
at 1 degree of freedom. We used the value of the latter to determine whether the association was weak
(Cramer’s V value between 0.1 and 0.3), moderate (Cramer’s V value between 0.3 and 0.5) or strong
(Cramer’s V value between 0.5 and 1.0). To visually represent the observed associations, we used mosaic
plots in which two-dimensional frequency table is displayed using rectangles. The rectangular regions
in the mosaic plot are proportional to the cell frequencies they represent, where shadings indicate the
residuals between observed and expected frequencies. Positive value of a residual is represented by the
blue color, and negative value of a residual is represented by the red color, whereas the shade of color
indicates the site of residual: the larger the absolute value of residual, the darker the color, and vice versa,
the smaller the absolute value of residual, the lighter the color. This is also presented in the legend added
to the mosaic plot (Kabacoff, 2015).
Results
We rst present the results of the manual coding (RQ1) and then the results of the analysis using
the Atlas.ti tool, which was used to produce linear word clouds for all analysed research reports together
(RQ2) and separately based on research reports from before the Covid-19 epidemic and two and a half
years later (RQ3).
Coding results
Based on a close reading of a selection of research reports (N=90), we identied 10 key content
categories (codes) in the texts: content, personal preference, reason, user experience, price, quantity,
basic, development, quality, advanced and endorsement.
Figure 1. Comparison of paradigm models with Rojko (2020)
www.ijcrsee.com
272
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
Figure 1 shows that our analysis produced slightly different results than Rojko (2020) as interaction,
motivation, recommendation, and traditional were replaced by reason, development, and content, in
Figure 1 right, according to our ndings. Similarly, Rojko (2020) found that quality dominates over user
experience and quantity, whereas our ndings showed a different picture – as presented in the Figure 2.
Figure 2. Representation of each dominant word by code
Figure 2 shows the representation of each of the dominant words by codes (We did not add the code
area to the selection (dominant words: coding, programming, foreign, language(s), technology) because
it is not relevant, as these are the elds of study of the students whose research seminar papers are
analysed.), which were written at least 200 times in all research reports combined. It turns out that content
stands out the most (words: content, tutorial, example, material, new, video, game, help, information,
data), followed by personal preferences (words: option, year, years, time, different, decision, way,
possibility, relevant, important, choose, criterion, need, work). For content, we would highlight the words
video, game, and help as they best convey the students’ observations, while for personal preferences we
would highlight time, different, and way as they best convey the students’ thinking.
Dominant words
The dominant words were extracted using Atlas.ti tool, but despite the translation of the analysed
research reports into English, a lot of manual renement was necessary (see Appendix 1 for a full list of
removed words). The words that appear in Figure 3 appear at least 200 times in the analysed reports. The
most frequently occurring word is language (N=1476; also, languages in 16th place (N=657)), which in turn
refers to the domains of the applications and platforms tested (computer languages, foreign languages).
This is also the case for the third-ranked word programming (N=1342), the twelfth-ranked word coding,
etc. The second-ranked word is knowledge (N=1419), which we have classied under the code reason.
The results based on the most frequent individual words thus show that knowledge is the most important
for students, research is needed, followed by content, time is a constraint, and free, variety, and quality
are also among the most important.
It should also be noted that it is necessary to group certain words that appear separately in the
gure below, e.g. research (N=1318) and search (N=235), information (N=595) and data (N=508), nd in
the present and past tense (N=559 and N=371), video (N=459) and videos (N=270), year (N=238) and
years (N=204), play (N=358) and games (N=205), etc.
www.ijcrsee.com
273
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
Figure 3. Dominant words appearing at least 200 times (all assignments).
The sentiment analysis provided by Atlas.ti, without manual cleaning of the results, shows that
negative sentiment prevails (2305 in paragraphs and 4131 in sentences) over positive sentiment (1756
in paragraphs and 2593 in sentences). But a quick look at the results shows that a lot of manual cleaning
would be needed, as the sentiments are very often improperly attributed. Therefore, to identify the
experience from positive to negative, we also focused on the most frequently written words. This revealed
that the positive experience of students’ testing educational apps and platforms is the predominant one,
and Figure 4 shows the words that were used most frequently to describe the experience. These words
are also included in Figure 3.
Figure 4. Dominant experience words from positive to negative, occurring at least 200 times (all
assignments)
As shown in the gure above, 81% (N=2,781) have a positive sentiment. We would add that the
word mediocre also appeared 215 times among the most frequent words, but we did not include it in
Figure 4. If we had evaluated it as a neutral sentiment, the positive sentiment would have dropped to
76%. The limitation is that the included words are subjectively classied as sentiment and that only words
that appear at least 200 times in all research reports are used, but the result is clear. Again, in personal
conversations with each student, we perceived an overwhelmingly positive attitude, although occasionally
they encountered some problems when taking the test.
www.ijcrsee.com
274
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
Comparison
Following the ndings presented above we also carried out a comparative analysis between the
periods before and after Covid-19 pandemic. In this context, we have referred to the research reports from
the rst half of the 2019/20 academic year as pre-pandemic period research reports, and the research
reports from the second half of the academic year 2021/22 as post-pandemic period research reports.
The analysis of the pre-pandemic period research reports considered 16 reports, and the analysis
of the post-pandemic period research reports considered 10 reports. They both involved a lot of manual
cleaning and Figure 5 shows the result – words that appear at least 50 times. Considering the pre-
pandemic period reports, if we remove the words related to the elds of study (programming, coding, and
language), we see that the most important are content, reason, and personal preferences, and the top
words within the codes are content (N=352), followed by knowledge (N=275) and research (N=166). Also
standing out are way, different, video, time, new, experience, nd, and information. Considering the post-
pandemic period research, we notice that the words relating to the elds of study do not appear rst in the
top positions (in fact, only language appears in the list, in the 15
th
position).
Figure 5. Dominant words appearing at least 50 times: (a) research reports from the rst half of the
academic year 2019/20 (pre-pandemic period reports), and (b) research reports from the second half of
the academic year 2021/22 (post-pandemic period reports)
However, in the post-pandemic period reports content is still the most important for students, while
personal preferences and reason changed their position, as personal preferences came as second most
important. Namely, the top words within the codes are content (N=311), followed by experience (N=177) and
knowledge (N=172). Rating, criteria, education, information, data, different, and decision also stand out.
This suggests that students became more focused on their own preferences when choosing educational
applications and platforms, likely because they became more acknowledged with the (enlarged) offering
compared to pre-pandemic.
The comparison was applied to the sentiment analysis as well. Figure 6 shows the words that
were used most frequently to describe the experience in pre- and post-pandemic period reports. We
acknowledge that od the sentiment in pre-pandemic period reports, good comes rst, followed by better,
then problem/issues. Positive sentiment dominates with 64%. In post-pandemic period reports, however,
sentiment has come to the front, with the words good and better appearing in the top ve.
www.ijcrsee.com
275
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
Figure 6. Dominant experience words from positive to negative: (a) research reports from the rst half of
the academic year 2019/20 (pre-pandemic period reports), and (b) research reports from the second half
of the academic year 2021/22 (post-pandemic period reports)
Thus, in post-pandemic period reports positive sentiment is 100%, as the words problem or
problems do not appear, which is one of the most important differences compared to the results of the
students’ test results two and a half years earlier. This is per our belief due to the changed attitude of
students towards this type of learning, but also to the rapid development of educational applications and
platforms, which has been accelerated by the pandemic. The word mediocre appears, which, if taken into
account, could reduce satisfaction with testing by 33%.
In order to support the ndings presented above, we also performed chi-square tests at signicance
level 0.05 where the strength of the association was determined by the value of the Cramer’s V contingency
coefcients. Figure 7 represents the mosaic plots for a) dominant words, and b) dominant experience, in
both cases comparing the pre- and post-periods periods.
Figure 7. Comparison considering (a) dominant words, and (b) dominant experience in the pre- and post-
pandemic period reports
To compare the pre- and post-pandemic period reports regarding the dominant words, we have
considered only those words that appear among the dominant ones in both periods. In this case we
observe signicant (p < 0.05) but weak (Cramer’s V = 0.184) association between pre- and post-pandemic
periods. The discrepancies between the two compared periods occur in the case of the words different,
experience, and knowledge, where in the post-pandemic period experience has signicantly gained in
www.ijcrsee.com
276
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
importance, while the importance of different and knowledge has declined signicantly as compared to
the pre-pandemic period.
Considering sentiment analysis, we observe strong (Cramer’s V = 0.482) and signicant (p < 0.05)
association between dominant experience in pre- and post-pandemic period reports, which is consistent
with the fact, that positive sentiment among post-pandemic research reports is 100%, as the words
problem or problems do not appear. This nding can most likely be attributed to the rapid development of
educational applications and platforms, and the accelerated spread of their use as a consequence of the
pandemic.
Discussions
The methodology of extracting the main ndings from the students’ research reports, where we
paid attention to the students’ opinions and observations without combining them with our views and
expectations, proved to be a difcult task but helped us to provide neutral answers to our research
questions. The content analysis of the students’ reports was carried out using manual coding and the
qualitative analysis tool Atlas.ti provided a detailed insight into the students’ opinions. Below we summarise
the main ndings that answered our research questions:
RQ1: What has proved to be the most important in the testing of educational apps and platforms
by students?
The most important category was content, where we would point out the following most frequently
written words: video, game (including tutorial, example), and help. This is followed by the category
of personal preference, as students supplement their education with educational apps and platforms
according to their possibilities, needs, and time, and variety and method are also important to them.
This is because nowadays, the diversity of learning needs, styles, and social backgrounds of students
are increasingly coming to the fore in education (Heacox and Pengal, 2019), which the second-ranked
personal preference code conrms. Reason, user experience, price, and other categories follow.
However, we expected that, although some apps and platforms provide certicates of knowledge,
the absence of ofcial recognition might demotivate students to learn online on their own initiative. But
we found that formal education is not the only criterion for some companies that also seek recruits via
e-learning platforms and applications (in the cases studied, these are so far only foreign companies), and
the qualitative analysis also resulted in the category of certicate appearing as a certicate in the word
cloud (N=303) being ranked last.
RQ2: Which words stand out in the research reports of students who have tested educational apps
and platforms?
We expected that the words quality, user experience, quantity, price, and certicate, and their
synonyms (as in Rojko, 2020) would appear most often. But it turned out quite differently. If we exclude
the most frequent words related to elds of study, the word knowledge ranks highest, which we have
placed under the category of reason, followed by research, content, time, free, variety, and quality. The
word certicate is ranked only 52
nd
.
It also showed that students’ experience of testing educational apps and platforms is predominantly
positive (81%), with the words better, good, useful, like, advantage and best being the most frequent,
while negative sentiment is characterised by the word problem or problems.
RQ3: Has the impact of the Covid-19 coronavirus pandemic changed the way students view
educational apps and platforms?
While the analysis of the research reports from the rst half of the 2019/20 academic year shows
that the most important is content, followed by knowledge, research, method and different, the reports
from the second half of the 2021/22 academic year the word content also ranks best, followed by good,
experience, knowledge and better. Thus, in the post-Covid-19 reports, sentiment has come to the
front, with 100% of these research reports expressing positive sentiment, as compared to 64% positive
sentiment in the reports from two and a half years ago. We also observe (considering the ranking of words
by code) that students now prioritise user experience, reason, and quality over content and personal
preferences compared to the pre-pandemic period but are still aware of the need for exploration. Namely,
the development of the World Wide Web and search engines has made information ‘on demand’ a reality
(Hirsh, 2018), and technology allows them to learn at their own pace and time (Moreno, 2018).
www.ijcrsee.com
277
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
Conclusions
Our results show that the development of ICT and the related rise of educational applications
and platforms is having a signicant impact on teaching and learning. This requires teachers to take
these applications and platforms into account and to use new teaching methods, the rst prerequisite
being a certain level of digital competence. However, we have found that this is in fact a huge burden
for the majority of teachers, as it requires additional and continuous research and learning. It is also
becoming increasingly apparent that traditional education must compete with e-learning opportunities,
or the profession may gradually become meaningless for many teachers. It has also become clear that
teachers, therefore, need to change their mindset and see their profession as adding value and building
on what students can learn online, while at the same time becoming facilitators of such learning.
Although most courses do not yet foresee the use of educational apps and platforms, when teachers
use some (at least occasionally) during their teaching, it can further motivate students to participate and
pay attention. In addition, teachers can recommend certain apps and online platforms for students to
use for self-learning to deepen and consolidate their knowledge. Our research showed that students
already know and use many of the apps and platforms, but it turned out that they also appeared to be
very interested in exploring and learning about others that exist in their eld of interest, as they have
certain advantages, disadvantages, and specicities, and they were always very interested to follow the
presentations of their classmates’ testing results.
Most students do not actively participate during a traditional lecture (Bajpai, Biberman and Ye,
2019), but educational apps require exactly that. Nevertheless, we would like to stress that we do not
believe that the integration of apps and online platforms in faculty courses is always necessary or that it
can always improve teaching and learning. However, in the spirit of the times, where students are used
to being constantly connected to the World Wide Web and using connected devices at the same time for
learning, searching for information, communicating, etc., we have decided to present the results of our
survey to give an insight into students’ opinions and testing results. In addition, this encouraged students
to think about the possibilities and options for lifelong learning that will be required of them, especially
intensively in the future career eld of the students whose research reports were the source of the data
of our empirical research. Therefore, this assignment was proposed to nal-year students in the eld of
computer science and informatics.
The limitations of our research are clear; we only present tests of a few online platforms and
applications, but the list can never be denitive as it is subject to constant updates. Moreover, depending
on the elds of study, we only present the ndings of a limited number of students, all from the four
Informatics and Computer Science degree programmes from one faculty from Slovenia. Finally, although
we do not assess the performance, usability, user experience, etc. of applications and web platforms, we
present the main ndings of students when testing them to gain insight into their thinking, our assignment
of codes is subjective, as is the manual cleaning of the analysis results by Atlas.ti.
Despite these limitations, the scientic contribution of our paper stems from the fact that other
authors have not used the same methodology to shed light on the topic of the paper. This topic is also
extremely interesting in the so-called post-coronal period, as it shows how quick the change was, due to
the coronal period and thus has important practical implications. Moreover, the empirical results of the
qualitative analysis of students’ opinions are particularly valuable, as their perspective is often neglected.
At the same time, we also conducted a personal interview with each student, which allowed us to code
and analyse the word cloud more appropriately.
Online education is certainly an area that will continue to grow rapidly in the future. Indeed, with
new technologies such as machine learning and 5G, we can expect further rapid growth. But here is the
problem – teachers and students do not know what is out there already, which limits them. Therefore,
lifelong learning also applies to teachers, who should also spend their working time exploring these
possibilities so that they know how to use them and recommend them to students as an additional
resource, and their role should focus even more on a mentoring style of teaching. However, this will only
be effective if students are committed to this way of learning, which will require them to be even more
self-controlled and self-organised.
During the Covid-19 coronavirus pandemic teachers and students experienced a situation where
distance teaching and learning became mandatory, requiring them to change their methods signicantly,
which further increased the use of educational applications and platforms, not to mention the various online
communication tools. As Kodelja (2020) states, it is difcult to say how successful distance education has
been as a result of the new coronavirus, but it seems that not all the possibilities offered by ICT have been
exploited, due to both objective and subjective circumstances. But due to this pandemic, we cannot go
www.ijcrsee.com
278
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
back to the learning and teaching methods we used before. We need to take a further step based on the
experiences we have gained and exploit all the possibilities offered by ICT, thoughtfully and adapted for
different groups of students.
Acknowledgements
We thank to all students who made their research reports after testing the educational applications
and platforms.
Conict of interests
The authors declare no conict of interest.
Author Contributions
Conceptualization, K.R. and N.E.; methodology, K.R., and N.E.; formal analysis, K.R. and
N.E.; investigation, K.R.; validation, K.R. and N.E.; visualization, K.R. and N.E; writing—original draft
preparation, K.R. and N.E.; writing—review and editing, K.R. and N.E. All authors have read and agreed
to the published version of the manuscript.
References
Adenusi, D. A., Adebayo, A. A., & Oni, B. O. (2019). ICT In Education Among Higher Education Students (A case study of The
Polytechnic, Imesi-Ile, Osun State). Villanova Journal of Science, Technology and Management, 1(1). Retrieved from
https://nigerianjournalsonline.com/index.php/VJSTM/article/view/348
Anderson, R. D. (2002). Reforming science teaching: What research says about inquiry. Journal of science teacher education,
13(1), 1-12. https://doi.org/10.1023/A:1015171124982
Bajpai N., Biberman J., & Ye, Y. Y. (2019). ICT for Education: Lessons from China. Center for Sustainable Development, Earth
Institute, Columbia University. ICT India Working Reports, 20. Retrieved from https://academiccommons.columbia.edu/
doi/10.7916/d8-5scg-7845
Berényi, L., & Deutsch, N. (2018). Effective teaching methods in business higher education: a students’ perspective.
International journal of education and information technologies, 12, 37-45. Retrieved from http://real.mtak.hu/90463/1/
a122008-018.pdf
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and
Accountability (formerly: Journal of personnel evaluation in education), 21, 5-31. https://doi.org/10.1007/s11092-008-
9068-5
Boh Podgornik, B., Dolničar, D., Šorgo, A., & Bartol, T. (2016). Development, testing, and validation of an information literacy
test (ILT) for higher education. Journal of the association for Information Science and Technology, 67(10), 2420-2436.
https://doi.org/10.1002/asi.23586
Bregar, L., Zagmajster, M., & Radovan, M. (2010). Osnove e-izobraževanja: priročnik [Fundamentals of e-learning: a handbook].
Andragoški center Slovenije.
Buijs, M., & Admiraal, W. (2013). Homework assignments to enhance student engagement in secondary education. European
journal of psychology of education, 28, 767-779. https://doi.org/10.1007/s10212-012-0139-0
Castañeda, L., & Selwyn, N. (2018). More than tools? Making sense of the ongoing digitizations of higher education. International
Journal of Educational Technology in Higher Education, 15(1), 1-10. https://doi.org/10.1186/s41239-018-0109-y
Data Bridge Market Research. (2022). Global Massive Open Online Courses (MOOCS) Market – Industry Trends and Forecast
to 2029. Retrieved from https://www.databridgemarketresearch.com/reports/global-mooc-market
Dhawal, S. (2021). By The Numbers: MOOCs in 2021. Retrieved from https://www.classcentral.com/report/mooc-stats-2021/
Grooms, L. D. (2018). Distance Teaching and Learning Platforms. In Encyclopedia of Information Science and Technology,
Fourth Edition (pp. 2455-2465). IGI Global. https://doi.org/10.4018/978-1-5225-2255-3.ch214
Heacox, D., & Pengal, Z. (2009). Diferenciacija za uspeh vseh: predlogi za uspešno delo z učenci različnih zmožnosti:
preizkušeni nasveti in zamisli za učinkovito poučevanje [Differentiation for Everyone’s Success: Suggestions for
Working Successfully with Students of Different Abilities: Proven Tips and Ideas for Effective Teaching]. Rokus Klett.
Hirsh, A. (2018). Technology on the run: Promoting active behavior in diverse ICT-supported physical education classes.
Studia Edukacyjne, 47, 343-359. Retrieved from https://repozytorium.amu.edu.pl/bitstream/10593/23779/1/SE%20
47_2018_Alon_Hirsh.pdf
Jagodič, M. (2010). E-izobraževanje na slovenskih univerzah [E-education at Slovenian universities] (Doctoral dissertation,
Univerza v Mariboru, Fakulteta za organizacijske vede). Retrieved from https://core.ac.uk/download/pdf/67540095.pdf
Kabacoff, R. (2015). R in Action: Data analysis and graphics with R. 2
nd
Edition. New York: Manning Publications Co.
Kodelja, Z. (2020). Šolstvo v času pandemije: izobraževanje na daljavo [Education in times of pandemic: distance education].
Sodobna pedagogika, 71(4), 42-56. Retrieved from https://www.sodobna-pedagogika.net/clanki/04-2020_solstvo-v-
casu-pandemije-izobrazevanje-na-daljavo/
Krastev, N. (2019). Why Udemy is great. Retrieved from https://medium.com/@nedko.krastev/why-udemy-is-great-
2d06f18804bc
www.ijcrsee.com
279
Rojko, K., & Erman, N. (2023). The impact of the Covid-19 pandemic on higher education students’ perceptions of educational
applications and platforms, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),
11(2), 267-279.
Lautenbach, G. (2014). A theoretically driven teaching and research framework: learning technologies and educational practice.
Educational Studies, 40(4), 361-376. http://dx.doi.org/10.1080/03055698.2014.910445
Margaret, R. E., Uma, B., Tejonidhi, M. R., & Neelakantappa, B. B. (2018). A Recipe for the Use of ICT Tools in an Educational
Institute. Journal of Engineering Education Transformations, 31(3), 114-119. https://doi.org/10.16920/jeet/2018/
v31i3/120778
Mohar, A. K., & Kovač, M. (2021). Digitalna učna gradiva po pandemiji kot del šolske rutine?. Sodobna Pedagogika, 72(138),
28-43. Retrieved from https://www.sodobna-pedagogika.net/clanki/01-2021_digitalna-ucna-gradiva-po-pandemiji-kot-
del-solske-rutine/
Moreno, R. (2018). Revista de la sociedad Española de materiales [Journal of the Spanish society of materials]. Retrieved
from https://www.researchgate.net/publication/328492871_REVISTA_DE_LA_SOCIEDAD_ESPANOLA_DE_
MATERIALES
Palir Mavrič, L. (2017). Uvajanje elementov formativnega spremljanja v pouk [Introducing elements of formative monitoring
into lessons]. Partnerstvo Pedagoške fakultete Univerze v Ljubljani in vzgojno-izobraževalnih inštitucij 2017, Book
of Proceedings, edited by M. Metljak, 73-84. Retrieved from http://www.pef.uni-lj.si/leadmin/Datoteke/Posvet/
Partnerstvo_Posvet-PeF-2017_zbornik-prispevkov.pdf#page=74
Panigagua, A., & Istance, D. (2018). Teachers as Designers of Learning Environments: The Importance of Innovative edagogies.
Educational Research and Innovation, Paris, OECD Publishing, http://dx.doi.org/10.1787/9789264085374-en
Pietarinen, T., Palonen, T., & Vauras, M. (2021). Guidance in computer-supported collaborative inquiry learning: Capturing
aspects of affect and teacher support in science classrooms. International Journal of Computer-Supported Collaborative
Learning, 16(2), 261-287. https://doi.org/10.1007/s11412-021-09347-5
Powell, N., Cleveland, R., Thompson, S., & Forde, T. (2012). Using Multi-Instructional Teaching and Technology-Supported
Active Learning Strategies to Enhance Student Engagement. Educational Leadership and Policy Studies, 4(2), 41-50.
Retrieved from https://encompass.eku.edu/cgi/viewcontent.cgi?article=1007&context=elps_fsresearch
Rojko, K. (2020). Innovative learning and teaching in higher education supported by web platforms and applications. Zbornik
radova Međunarodne naučne konferencije o digitalnoj ekonomiji DIEC, 3(3), 67-80. Retrieved from http://ipi-akademija.
ba/le/diec-2020-zbornik-b5-online-sa-naslovnom-1/117
Semerci, A., & Aydin, M. K. (2018). Examining High School Teachers’ Attitudes towards ICT Use in Education. International
journal of progressive education, 14(2), 93-105. https://doi.org/ 10.29329/ijpe.2018.139.7
Sinha, S., Rogat, T. K., Adams-Wiggins, K. R., & Hmelo-Silver, C. E. (2015). Collaborative group engagement in a computer-
supported inquiry learning environment. International Journal of Computer-Supported Collaborative Learning, 10, 273-
307. https://doi.org/10.1007/s11412-015-9218-y
Suwal, S., & Singh, V. (2018). Assessing students’ sentiments towards the use of a Building Information Modelling (BIM)
learning platform in a construction project management course. European Journal of Engineering Education, 43(4),
492-506. http://dx.doi.org/10.1080/03043797.2017.1287667
Urbančič, M. (2021). Programirani pouk in učno oziroma poučevalno okolje. Sodobna pedagogika, 72(4), 138-150. Retrieved
from https://www.sodobna-pedagogika.net/en/archive/load-article/?id=1990
Webb, N. M., Franke, M. L., Ing, M., Turrou, A. C., Johnson, N. C., & Zimmerman, J. (2019). Teacher practices that promote
productive dialogue and learning in mathematics classrooms. International Journal of Educational Research, 97, 176-
186. https://doi.org/10.1016/j.ijer.2017.07.009
Wise, J. (2022). Elearning Statistics 2022: Market Size & Industry Growth. Retrieved from https://earthweb.com/elearning-
statistics/
