Introduction

In real life, theoretical knowledge and practical knowledge complement each other. While theoretical knowledge provides a fundamental understanding of concepts, practical knowledge enables individuals to implement and utilise those concepts in real-world situations. This relationship between theoretical and practical knowledge is equally important in academics, as students need both theoretical knowledge and practical experience to apply the concepts they learn during their school or college study. By gaining practical experience, students can develop a deep understanding of the subject matter and develop critical thinking skills that could be applied to real-world problems (Naik & Gajjar, 2023).

However, it is not enough to assign projects, assignments and lab exercises. It is also crucial to evaluate whether students have truly understood the practical applications of the theoretical concepts. This is where practical examinations play an important role. To test whether students have truly understood the practical problem and are able to solve it correctly, a practical exam could be conducted that throws a challenging problem in front of them which needs to be solved within the stipulated time. These exams help students develop critical thinking skills, handle pressure and provide solutions within a given time frame, which ultimately boosts their confidence. Additionally, faculty can evaluate whether practical understanding is achieved or not and improvise on their teaching methods.

Due to COVID-19 and the rise of online classes and examinations, the concept of open-book examinations has gained popularity. However, there has always been a debate about its efficiency (Williams & Wong, 2009) and whether it should be promoted.

The open-book examination has its own pros and cons, but it helps students to focus on gaining a better understanding of theoretical concepts rather than just memorising them. The open-book examination also eliminates the stress of memorising. On the other hand, there is a concern that students may become casual towards learning and may not put in enough effort to truly understand the concepts, resulting in ineffective learning.

Overall, the effectiveness of open-book examinations depends on the difficulty level of the problem statement, the students’ approach to addressing the problem and their dedication towards learning/doing.

Literature Survey

Quite a bit of research has been conducted on the practical examination and its mode of conduct, especially after the global COVID-19 pandemic brought it into the limelight. There are studies which focus on various aspects of open-book examinations. The details are provided below.

One study (Brightwell et al., 2004) aimed to test the assumption that open-book examinations provide a better learning environment for students. A student in an anatomy and physiology course took an online multiple-choice exam in class and then took the same exam again with their textbooks. The results showed no significant difference in scores, suggesting that well-designed questions can assess student abilities in both open- and closed-book environments.

However, in another study (Feldhusen, 1961), the author discusses and analyses the value of the student ratings for the open-book examination and its relation to the examination scores. Also, he concludes that the open-book examination measures different abilities than the closed-book examination.

A study conducted by Block (2012) describes the use of open-book tests, closed-book tests and notecards in tests for an introductory statistics course. A review of the literature shows that open-book assessments are universally recognised to reduce anxiety. However, the literature is mixed on whether deeper learning or better preparation occurs with open-book exams. The study reviews the testing policy of the Math300 Statistics course, which evolved from closed-book exams to open-book exams to closed-book exams with notecards. Based on the author’s experience, it led to increased student enjoyment of the course while continuing to encourage deeper student learning.

Another research study (Ashri & Sahoo, 2021) describes the impact of the COVID-19 pandemic, the way it disrupted the higher education sector and the way it forced Indian academic institutes to adopt new methods for evaluating students. While many universities in India chose not to hold exams due to public health concerns, the University of Delhi conducted an online open-book examination (OBE). The article evaluates the strengths, weaknesses, opportunities and challenges of online OBEs by drawing details from a wide range of literature. Additionally, the authors of this study compared student performance in OBEs and closed-book exams. They found that students generally score higher in OBEs compared to closed-book exams.

As per the findings of the study by Theophilides and Koutselini (2000), the students preparing for a closed-book examination tend to procrastinate their study at the end of the semester, concentrate on the assigned texts and memorise information. However, students preparing for an open-book examination tend to consult various sources and associate the information acquired. When taking the exam, they work productively, while at the same time probing deeply into the knowledge gained.

Furthermore, open-book practical examination and its importance is not only restricted to computer science subjects such as database management systems (Naik, 2014), computer networks and computer vision but is also applicable to the fields of nursing (Johanns et al., 2017), medicine (Zaguryorly & Durning, 2021), aviation and many more.

As a whole, the literary analysis broadened the perspective of this study and also helped in approaching this research. Additionally, it had a significant influence on the choices that were made as part of this primary study.

Data Analysis

Quantitative Survey

About the Data

The quantitative survey was conducted for university undergraduate students with majors in computer science and engineering who were asked to fill out a questionnaire based on their experience with practical examinations. A total of 80 responses were collected using Google Forms. Out of these, 78 students appeared in the practical examinations. These 78 responses represent the population size for performing quantitative data analysis.

For quantitative data analysis, four hypotheses were formulated and tested using statistical methods. These hypotheses are as follows and are discussed in detail in the section ‘Hypotheses Testing’.

1.     There should be no practical examination.

2.     It is easy to score in the open-book examination.

3.     Practical knowledge helps to solve real-life problems.

4.     The open-book practical examination is as good as the closed-book examination to test one’s understanding of the concepts.

Data Distribution and Visualisation

Out of 78 students, 56.4% (44) confirmed that they had an open-book practical examination and 43.6% (34) confirmed that they had a closed-book practical examination. Thereby, to avoid the biases generated because of the absence of closed-book examination takers in the population set, a sample dataset was created using random sampling. The sample size consisted of 66 students, with 33 students who took an open-book practical exam and 33 students who took a closed-book practical exam.

Based on the quantitative survey, 86% (57) of the students strongly believe that the practical evaluation component (assignments, labs and practical assessments) provides more hands-on experience of the theoretical concepts. However, only 3% (2) of the students decline this belief, and the remaining students are not sure about whether the practical evaluation component provides them with more hands-on experience with the theoretical concept.

The quantitative survey also indicated, as shown in Figure 1, that 92% (61) of the students agreed that practical knowledge of theoretical concepts helps them to solve real-life problems or is helpful in their professional life. However, about 6% (4) of students are neutral about the statement, and 2% (1) totally disagree with it.

Figure 2 shows that the majority of the participants believe that a practical examination is a good way of testing the practical understanding they have gained in the course.

The participants were asked to express their thoughts about the open-book examination, and Figure 3 shows that 68% (45) believe that even if the notes are available, a person could not answer the question correctly if he or she does not know the concepts. Furthermore, 54% (36) believe that an open-book exam can help to solve syntax errors.

 

Figure 1. Findings About the Usefulness of Practical Knowledge in Solving Real-life Problems.

 

Figure 2. Should There Be a Practical Examination?

 

Figure 3. Thoughts About an Open-book Practical Examination.

 

Figure 4. Preferred Mode of Examination.

 

Figure 4 shows the student preferences for the mode of examination. As per the figure, 68% of them preferred an open-book, and 32% preferred a closed-book practical examination.

Qualitative Survey

As part of the qualitative survey, a total of six interviews were conducted to gather qualitative data. Each participant’s transcript was initially coded independently, providing context for the raw data in relation to the research questions. Groups were created based on individual coding and many similarities, as well as some unique qualities, were found in the transcripts written after the interviews. Based on responses from participants in the qualitative survey, additional information from the participants was collected. The common trait among the participants was a lack of awareness about the importance of practical knowledge. The survey was not limited to students who took only open-book practical exams. Another trait was their casual approach and shyness to share details, but during the verbal survey, they were confident and honest in discussing their experiences of the open-book practical exam.

The majority of students are willing to take an open-book practical examination. They believe that this can be a good way to check their practical understanding, which they have acquired from theoretical knowledge. However, they are also concerned about the possibility of unfair means being used during online open-book examinations, such as sending answers to friends via various platforms. Additionally, they mentioned that open-book examinations can relieve the pressure of memorising concepts and they can concentrate on understanding concepts. A handful of people also mentioned that due to open-book examinations, students are solely responsible for their grades, and there will be no blame on the system for their low scores.

Results/ Findings of the Analysis

Quantitative Survey

Result Visualisation

Over the years, it was believed that scoring in open-book practical examinations would be easier than in closed-book examinations. However, after conducting data analysis and visualisation, it was found that even with an open-book examination, achieving a high score is not a simple task. In fact, over 50% of respondents indicated that they did not find this to be true.

Thereby, to find the credibility of the responses, further analysis was done by comparing the marks achieved in a closed-book versus an open-book practical examination. Figure 5 depicts that the average score of closed-book examinations was higher than that of open-book examinations. Hence, providing evidence gives the insight that it is not easy to score in an open-book examination.

The graph in Figure 6 shows that having a lower score implicitly implies satisfaction with an examination score, which will be lower when it is an open-book examination. However, it was the opposite and it was found that the majority of them were satisfied with scores and did not have more than a 7% difference in the satisfaction level of open-book and closed-book scores, as shown in Figure 7.

 

Figure 5. Is it Easy to Score in a Closed-book Examination?

 

Figure 6. Close-book vs. Open-book Practical Examination Score.

 

Figure 7. Satisfaction of Close-book vs. Open-book Practical Examination Score.

 

Therefore, to understand the reason for being satisfied despite a low average score, it can be inferred from Figure 8 that, even though there is an open-book examination, it is not easy to score in that if your concepts are not clear and you are not dedicated or serious about learning them. So, to assert this thought, the overall percentage was quantified as students’ dedication level and mapped with a practical score to understand whether if a student is dedicated, then they can score better irrespective of the mode of examination.

Hypothesis Testing

Based on the quantitative survey, respondents were asked whether practical examinations were a good way to check their practical understanding. Using the data points, the above hypothesis was formulated. Based on the chi-square test results shown in Table 1 with a p value less than the significance level of .05 (Naik & Purohit, 2023), the null hypothesis was rejected. Therefore, the research question ‘Should there be a practical examination?’ was answered in the affirmative, as there should be a practical examination.

 

Figure 8. Dedication vs. Practical Exam Score.

 

Table 1. Hypothesis Testing: Should There Be a Practical Examination?

Note:

H₀: There should be no practical examination.

H₁: There should be a practical examination.

 

According to the results of the quantitative survey, it was analysed whether test-takers find it easier to get good scores in open-book examinations compared to closed-book examinations. Using the gathered data, the hypothesis mentioned above was formulated. Based on the chi-square test results shown in Table 2, with a p value less than the significance level of .05, the null hypothesis was rejected. Therefore, the answer to the research question ‘Is it easy to score in an open-book examination?’ is that it is not easy to score in an open-book examination.

The research investigated whether test-takers believed that applied knowledge is necessary to solve real-life problems. This led to the hypothesis that practical knowledge is required. The results of the chi-square test shown in Table 3 reject this hypothesis, with a p value less than the significance level of .05, thus answering the research question ‘Is practical knowledge required to solve real-life problems?’ with confidence.

 

Table 2. Hypothesis Testing: Is it Easy to Score in an Open-book Examination?

Note:

H₀: It is easy to score in an open-book examination.

H₁ = It is not easy to score in an open-book examination.

 

Table 3. Hypothesis Testing: Is Practical Knowledge Required to Solve Real-life Problems?

Notes:

H₀: It is not easy to score in an open-book examination.

H₁: It is easy to score in an open-book examination.

 

Based on the results of a quantitative survey, further investigation was conducted to determine whether test-takers prefer open-book practical examinations. Using this data, the aforementioned hypothesis was developed. After conducting the chi-square test, the null hypothesis needs to be accepted since the p value is greater than the significance level of .05. Therefore, the inference shown in Table 4 can be drawn: open-book practical examinations may be preferred, answering the research question ‘Should open-book practical examinations be given preference?’

Qualitative Survey

After conducting one-on-one interviews and reviewing the notes taken during the interviews, the following three themes were developed. These themes serve as the core concepts of the research and allow for analysis and the provision of various perspectives from the interviewees. These perspectives are essential to the research.

 

Table 4. Hypothesis Testing: Should an Open-book Practical Examination Be Given Preference?

Notes:

H₀ = The open-book practical examination is as good as the closed-book examination to test one’s understanding of the concepts.

H₁ = The open-book practical examination is not as good as the close-book examination to test one’s understanding of the concepts.

 

Practical Examination Is a Good Way to Check the Applied/Hands-on Knowledge

Based on the interviews conducted, the common inference drawn was that practical exams are valuable for validating understanding, while project-based learning and labs are effective in developing practical skills that are crucial for real-life problem solving. Participants emphasised the significance of practical understanding and critical thinking skills taught by practical exams. Additionally, exams improve the ability to work under pressure and solve problems within time constraints. All participants agreed that practical exams are necessary to accurately assess skills. The interviews highlighted the importance of project-based learning and labs in preparing individuals for real-world problem solving. Therefore, based on the interviews conducted, it is evident that practical exams, project-based learning and labs are essential for developing practical skills. These skills are crucial for individuals to excel in real-world problem-solving scenarios.

Pros and Cons of an Open-book Examination

After conducting interviews with various students, it can be concluded that online examinations have both advantages and disadvantages. Although online exams have become increasingly popular in recent years, they have become even more so after the global pandemic that forced many educational institutions to shift to remote learning and assessments. Interviews revealed that open-book exams are less stressful, especially practical exams, as students do not have to memorise syntax and other theoretical concepts, which may not be as important. This feature of open-book exams can be particularly helpful for students who struggle with memorisation or have test anxiety. Additionally, students are solely responsible for their marks. If they are unable to achieve the desired results, they have no one to blame but themselves. This can be motivating for some students, as it can encourage them to take ownership of their learning and to work harder to achieve their goals. Lastly, it could avoid mental health problems due to test anxiety or stress. However, there is a downside to open-book/online exams. The exams that allow students to access the internet increase the possibility of unfair means during the examination. This can create biases against other students who may not have access to the same resources or who may not be as technologically savvy. In addition, there are concerns about cheating and academic integrity. However, this issue can be addressed by increasing the number of invigilators to monitor students during the examination. This can help to ensure that students are following the rules and that the exam is fair for all participants. Lastly, sometimes students might get relaxed due to open-book examinations and do not put effort into learning concepts if they are aware from the start of the curriculum that the exams will be open-book. Hence, it was advised not to declare from the very beginning. In summary, open-book exams can be beneficial for some students, but there are also potential drawbacks that must be considered and addressed. It is important for educators to weigh the pros and cons of open-book exams and to consider the needs of their students when deciding whether to implement this type of assessment. Overall, open-book exams have the potential to be a valuable tool for assessing student learning, but it is important to ensure that they are conducted in a fair and ethical manner.

The Mode of Practical Examination Does Not Play Any Role in the Marks Achieved

After conducting and analysing the qualitative interviews, it became clear that a practical examination should be conducted in addition to the theoretical exams. The qualitative interviews highlighted that practical exams are essential to assess how efficiently and effectively students can implement their theoretical understanding to solve real-life problems within a time constraint. On the other hand, the mood of the examination does not seem to matter as much as the nature of the exam. Students believe that even if a book is given, it would be time-consuming and difficult to solve the problem within the given time constraint if they had not studied throughout the semester or year. Therefore, the mood of the examination does not matter as much as the nature of the test. To solve problems in the examination, students need to go through notes or a textbook and be aware of each concept’s location. While an open-book exam may seem like a good idea, in reality, it may not be practical if the student has not been dedicated throughout the semester or year of study.

Hence, practical exams are essential in addition to theoretical exams, and the mode of the examination does not matter as much as the nature of the test. The student’s ability to implement their theoretical understanding and solve real-life problems within a time constraint is the most crucial factor for a successful exam.

Conclusion

After conducting both qualitative and quantitative research in this area of study, significant insights were gathered, and several research questions were answered. One key finding was that students are well prepared and eager to take practical examinations, as they consider it an important evaluation criterion. The mode of examination does not seem to be a significant factor in determining student performance.

Instead, their dedication and curiosity play a major role in achieving good marks, according to both students and policymakers. Therefore, it is recommended that educational institutes conduct practical examinations, with the mode of examination left to their discretion. However, it is worth noting that the research found that students generally prefer an open-book examination. This format provides them with the opportunity to access relevant materials during the exam, which they believe can help them perform better. It is the instructor’s responsibility to ask the questions in such a way that students can get help for syntax errors, not for logic or problem solutions, from the allowed sources/references/books. In conclusion, our research suggests that educational institutes should focus on encouraging and fostering student dedication and curiosity to ensure good academic performance.

Acknowledgment

The authors are grateful to the anonymous referees of the journal for their extremely useful suggestions to improve the quality of the article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Shefali Naik  https://orcid.org/0000-0002-0785-4282

Jap Purohit  https://orcid.org/0009-0005-9350-7672