Designing open-book exams
Last updated: November 3, 2023, 12:58 p.m.
Guidelines for creating open-book exams
Open-book and take-home exams are a method of testing, usually in an unsupervised environment, that allows students to use textbooks, class notes, memory aids and other reference material to complete the exam. These exams can assess a range of competencies, but are especially useful for evaluating a student’s ability for higher order of thinking over their ability to recall factual information.
Many essay questions and short answer questions already target the application of higher-order thinking skills and can be readily used as an open-book or take-home exam with minor revisions. Questions that test ‘recall’ are typically better suited to multiple choice exams, so will need to be revised to become effective open-book exam questions.
Target higher-order thinking skills
Higher-order thinking references the different levels of knowledge according to Bloom’s Taxonomy of Knowledge. Depending on your learning outcomes and where your course is situated within your program, you will be targeting different levels of discipline-specific knowledge and concepts with your open-book exam questions. Below is a summary of lower and higher order thinking skills and associated question words.
- Questions that target application and ask students to solve, use, examine, compare, contrast, relate, etc.
- Questions that target analysis and ask students to infer, explain, differentiate, distinguish, relate, etc.
- Questions that target synthesis and ask students to develop, organize, design, create, integrate, etc.
- Questions that target evaluation and ask students to judge, critique, justify, recommend, assess, etc.
Important note: Students should be well accustomed to higher-order question types prior to taking the exam.
Tips for writing open-book exam questions
- Clearly define the learning outcome you want to assess. In other words, your questions should measure the skills or knowledge that students acquired as part of your course.
- Keep the number of questions realistic and in line with the time limit of your exam and the grading required.
- Situate your question within a particular context, problem or situation. Ensure the context, problem or situation is clearly outlined for students.
- Formulate a question that requires students to demonstrate their thinking related to their discipline knowledge / the course concepts (E.g., Apply their knowledge of a theory to your context, problem or situation. *This will ensure they are using higher order thinking skills rather than simply locating and summarizing the relevant information to answer the question).
- Construct the question so that the task is clearly defined for students. Avoid passive sentence structure and ambiguous phrasing.
- Use directive verbs to clarify the types of thinking and content you require in the response. Refer to the directive verbs in the section on Higher-order question types above.
- Keep the number of questions realistic (with open book exams, we might be tempted to give more questions)
- Validate your questions by having a TA or colleague review them for clarity and alignment with the intended assessment outcomes.
Note: Avoid using vague terms like “discuss” as this can lead to responses that are too broad or vague.
1. Send instructions to students about how to prepare for an open book exam well in advance (suggested materials for review, what to focus on, what kinds of responses will be expected, etc.)
2. Give students more time to complete the open book exam than a traditional exam.
- Consider a good guideline to be: half of the time spent on finding information and planning and the other half for writing up answers.
3. Have a clear grading criteria and let students know in advance.
4. Determine how you will administer and collect the exam.
- You can collect essay-type questions through Moodle Assignment or you can use Moodle Quiz to administer shorter answer exams or generate random essay questions for each student. Either way, ensure students have done it before. For example, if students have never submitted assignments to Moodle or used Moodle quiz before, make sure to provide a practice activity that follows the same procedure.
5. Determine the timing of the Exam. Will you make it available only for the exact amount of time needed to complete it, or will you give them a longer period (e.g., a 12-hour or two-day window, etc.)? You will need to configure Moodle accordingly and let students know well in advance.
Preventing plagiarism
A systematic review (Bengtsson, 2019) described a range of remedies for decreasing plagiarism on non-proctored open-book and take-home exams. A summary of these remedies follows.
- Design questions so that they require a thorough understanding of course material so that they will be costly to "contract out" (Bredon, 2003).
- Ask for proof and justifications for all answers (Lopez et al., 2011; Svoboda, 1971).
- Introduce an honour code (Fernald & Webster, 1991; Frein, 2011).
- Grade down for answers without a reference (Freedman, 1968).
- Require answers to make direct references to course-specific material (Williams & Wong, 2009).
- Make questions “highly contextualized” (for example, from the perspective of a particular character, during a certain time period, the impact in very specific circumstances/places; Williams & Wong, 2009).
- Randomly scramble the order of questions and answers (Frein, 2011; Tao & Li, 2012; Murray, 1990). (This is for multiple choice type questions. There is an option in Moodle Quiz for this.)
- Narrow the timeframe to complete the test (Frein, 2011; Lancaster & Clarke, 2017).
- Use a security browser to prevent printing and saving of the exam (Tao and Li, 2012).
- Ask for hand-written answers (you can ask students to take a picture and upload this; Lopez et al., 2011).
References
- Bengtsson, L. (2019). Take-home exams in higher education: A systematic review. Education Sciences, 9(4), 267.
- Bredon, G. (2003). Take-home tests in economics. Economic Analysis and Policy, 33(1), 52-60.
- Freedman, A. S. (1968). The take‐home examination. Peabody Journal of Education, 45(6), 343-347.
- Frein, S. T. (2011). Comparing in-class and out-of-class computer-based tests to traditional paper-and-pencil tests in introductory psychology courses. Teaching of Psychology, 38(4), 282-287.
- Fernald, P. S., & Webster, S. (1991). The Merits of the Take-Home, Closed Book Exam. Journal of humanistic Education and Development, 29(4), 130-42.
- Lancaster, T. & Clarke, R. (2017, May 24–26). Rethinking assessment by examination in the age of contract cheating. In Proceedings of the Plagiarism Across Europe Beyond, 215-228. ENAI: Brno, Czech Republic.
- Lopéz, D., Cruz, J.-L., Sánchez, F., & Fernández, A. (2011, October 12–15). A take-home exam to assess professional skills. In Proceedings of the 41st ASEE/IEEE Frontiers in Education Conference. Rapid City, SD, USA.
- Svoboda, W. S. (1971). A case for out-of-class exams. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 46(4), 231-233.
- Tao, J., & Li, Z. (2012). A case study on computerized take-home testing: Benefits and pitfalls. International Journal of Technology in Teaching & Learning, 8(1), 33–43.
- Williams, J. B., & Wong, A. (2009). The efficacy of final examinations: A comparative study of closed‐book, invigilated exams and open‐book, open‐web exams. British Journal of Educational Technology, 40(2), 227-236.
Additional resources
- Best Practices: Open Book Exams developed by Ryerson University
- Transforming your current exam questions into open-book questions created by the University of Saskatchewan
- Open-book Examination, Assessment Resources developed by University of Hong Kong.
- Take-Home Exams in Higher Education: A Systematic Review by L. Bengtsson
- A guide for academics - Open book exams developed by the University of Newcastle Australia