Writing quiz and exam questions
Sections
- Questions that demonstrate understanding
- General tips for writing questions
- Tips for creating Multiple Choice questions
- Tips for creating Multiple Response questions
- Tips for writing Performance-Based questions
- Tips for writing Scenario-Based questions
- Key verbs and generic stems
It is important to write quiz and exam questions that are clear and consistent so that students learn but are not confused or tricked by the question. The best questions are performance-based questions. The same best practices apply when writing questions for a quiz or writing questions for a certification exam. If you happen to be responsible for both creating quiz questions and writing exam questions, be careful not to use the exact same question on a quiz and on the exam. They can be similar but must be different.
Questions that demonstrate understanding
Test items should always follow a consistent design so that the questioning process does not add unnecessary difficulty to answering questions. In addition, a logical and consistent format for writing test items also helps expedite the process of writing test items and provides a format for asking basic questions.When standard formats are used, test takers can quickly read and understand the questions since the format is expected. For example, to measure understanding of knowledge or facts, questions can begin with the following:
- What best defines....?
- What is (un)characteristic of...?
- What is an example of...?
General tips for writing questions
Tip 1
Questions must be clearly worded so that test takers comprehend the nature of the information being requested. In order to facilitate understanding, phrase the item so that the required answer is brief and sporadic. In addition, provide clear clues to indicate the expected response.
For example:
| Do this | Not this |
|---|---|
| An insurance company provides personal property insurance. Jewelry is categorized as a special type of personal property. Requests for jewelry coverage require that a user managing the request is trained to appraise jewelry. The requirement is to route assignments for jewelry appraisals to users who have this skill. Which one of the following routing approaches would you use? | An insurance company provides personal property insurance. Jewelry is categorized as a special type of personal property. Which one of the following routing approaches would you use? |
Tip 2
To prevent confusion and make scoring precise, phrase questions so there is only one answer or a limited range of answers that are possible. If multiple answers will correctly complete the item, make sure the number of correct responses is identified.
At the end of the question, write the following as shown: (Choose X) where X is the number of answers.
For example:
| Do this | Not this |
|---|---|
| Select the three true statements about the use of a BLOB field. (Choose Three) | Select the three true statements about the use of a BLOB field. (Choose 3); or Select the true statements about the use of a BLOB field. |
Tip 3
Utilize clear, explicit instructions that specify the format of the target answer (one word, multiple words, and so on) as well as the amount of acceptable variation (spelling, synonyms, and so on).
For example:
| Do this | Not this |
|---|---|
| An insurance company provides personal property insurance. Jewelry is categorized as a special type of personal property. Requests for jewelry coverage require that a user managing the request is trained to appraise jewelry. The requirement is to route assignments for jewelry appraisals to users who have this skill. Which one of the following routing approaches would you use? | An insurance company provides personal property insurance. Which one of the following routing approaches would you use? |
Tip 3
Limit the influence of extraneous clues to the correct answer by utilizing correct, neutral grammar. Avoid providing grammatical clues to the correct answer (plurals, "a" versus "an", specific modifiers, and so on) and make certain that all correct responses can fit grammatically in the blank.
For example:
| Do this | Not this |
|---|---|
| ______ is a template for creating _____ at run time. (The correct answer is Case type and cases; note that there are no grammatical clues before the terms.) | An _____________ is a concrete class (instance of the Rule-Obj-Class rule type) that corresponds to a table in an external relational database, rather than to a table or view in the PegaRULES database. (The correct answer is external class. Note that the article "an" signals that the correct answer begins with a vowel.) |
Tip 4
To reduce the emphasis on rote memorization of trivial information, do not use direct quotes from a text or lecture. Rather, phrase short answer items using unique or novel wording. This aids in evaluating high order thinking and problem solving; making associations to the material that are better for retention.
For example:
| Do this | Not this |
|---|---|
| The risk premium for a property insurance quote is based upon the location, size, and age of a house. Which two types of decision rules are best suited to configuring this decision? (Choose Two) | A decision rule is usually called from a flow and is used to make a decision about the path the flow should take. It can also be called from a declarative rule. The risk premium for a property insurance quote is based upon the location, size, and age of a house. Which two types of decision rules are best suited to configuring this decision? (Choose Two) |
Tip 5
Never use:
- None of the above
- All of the above (answers are randomized and you cannot lock an option are the bottom)
Use present tense, active voice.
For example:
| Do this | Not this |
|---|---|
The design document of an application states that the application must be implemented with UI responsiveness. Which behavior does the design require for the application?
| The design document of an application stated that the application should be implemented with UI responsiveness. Which behavior did the design require for the application?
|
Tip 6
Keep answer "clues" out of the question.
For example:
| Do this | Not this |
|---|---|
In which of the following cases would you use a paragraph? (Choose Two)
| If you have reusable pieces of text or need to display rich text, in which of the following cases would you use a paragraph? (Choose Two)
|
Tip 7
Distractors are:
- Clearly not the correct answer when the best answer is selected
- Plausible to someone who does not possess the skill being measured
- Incorrect answers that contain likely mistakes an unskilled person would make
- Independent and mutually exclusive — two or more distractors should not mean the same thing
Tips for creating Multiple Choice questions
A multiple choice question is constructed of an item stem, an answer, and three more distractors. There is only one correct answer for this format and may be written so that it measures not only knowledge of facts, but can also be used to evaluate high order thinking and problem solving.
When constructing the item stem (question), correct response, and distractors, consider the following:
- Relevance – Aligns directly with the competency being assessed. The item should focus on an area the test taker is likely to encounter in their role related to certification. Avoid items that simply test the test taker's memorization skills.
- Straightforward – Use plain English and present information in a clear and concise manner. Keep the global application of the certification in mind as well.
- Clarity – The stem and options must be clear and limited only to the information that the test taker requires to answer the question correctly. Generally, the longer the option the more likely it will end up providing hints to the correct response.
Tips for creating Multiple Response questions
A multiple response question consists of an item stem and more than one correct answer. Essentially, this is a combination of two or three multiple choice items in one. It is generally more difficult to answer and also discriminates very well between those who are proficient and those who are not in the given subject.
When constructing a multiple response question, consider the following:
- Plan for two correct responses out of five choices or three correct out of five or six choices. For example: What is the meaning of life? (Choose Two)
- Do not use "Choose all that apply." Instead, identify the number of choices that are needed to supply a complete and correct response. It is important to provide the number of correct choices required as a matter of fairness.
Tips for writing Performance-Based questions
Performance-based questions are related to tasks that must be done or specific problems that must be solved. Since we are unable to have a simulated environment, we recommend that these questions be based on scenario. Below are some examples of performance-based questions.
Tips for writing Scenario-Based questions
A scenario-based question (also known as a Reading Passage) provides a series of statement that outlines a scheme or sequence of events in summary or in synopsis form, followed by a question concerning what should be discovered or deduced from the scenario.
A scenario-based question lays out the major and minor premises so that the test taker can draw inferences based upon what they know in the subject area.
A scenario question presents reasoning from the general to the specific within a deductive reasoning process. The deduction leads to selecting the correct answer choice(s). This item format is very useful, especially for problem solving or troubleshooting types of questions. Scenarios can be written to describe a situation and ask the test taker to write out the correct answer-based on the information provided.
Scenario questions assess key proficiencies that are outlined by test objectives. Scenario questions should represent one or more of the following to evaluate higher level thinking.
- Identifies reasonable points of comparison
- Identifies evidence, key concepts, and principles
- Describes facts, situations, and predicted outcomes
- Identifies cause-effect or other critical relationships
- Recognizes how inferences are related to what is given
- Demonstrates clear purpose for grouping of structural components
- Demonstrates sound extensions for generalizations that fit specific situations
- Identifies, collects, and analyzes related evidence to possible outcomes or consequences
Ideas for Scenario Development
- What problems or situations might a developer encounter in every day work?
- What functions, roles, concepts, and abilities are essential components of a competent person performing this task?
- What situations might a less-than competent person cause or where might an improper sequence or lack of knowledge lead to an undesirable outcome?
Key verbs and generic stems
| Mental Behavior | Key Verbs in Student Outcomes | Generic Item Shell (Stems) |
| Recall | Who, what, when, where, which | Not recommended for Pega Certification Exams |
| Understanding | Define, demonstrate, describe, find, exemplify, explain, illustrate, list, listen, paraphrase, provide, show, tell | Which is the best definition of? Which is the correct definition for? Which is a characteristic of? What is the main symptom of? Which is the reason for? Which is an example of? Which is the relationship between? Which distinguishes? |
| Problem Solving | Answer, compute, calculate, determine, find, figure out, locate, solve | What is the problem? What are aspects of the problem? Solve for "x" Find a short cut How would you solve the problem? |
| Evaluating | Analyze, appraise, assess, attack, classify, compare, conclude, contrast, critique, defend, differentiate, distinguish, evaluate, relate, synthesize, value; Then choose, decide, judge, select, most appropriate, most effective, best | Which is the most effective? What is the difference between? What is a similarity between? Which of the following principles or procedures best applies? |
| Predicting | Anticipate, apply, conclude, deduct, induct, infer, hypothesize, predict, recommend, speculate, what happens, what if, then. Deductive reasoning infers unstated consequences from given principles and it creates a generalization from information. You are given a generalization and you are require to explain the evidence. It describes logical consequences. Deductive conclusions are absolute. Inductive reasoning infers to unknown generalizations from observations. You are given the evidence and are required to derive the generalization. | What would happen if? What is the main consequence of? What is the main cause of? What is most likely to happen? Based on the following principle, what prediction, conclusion, and hypothesis can you draw? What will happen next? This involves the use of an absolute or probable principle to anticipate. Based on the following facts or observations, what can you conclude? How likely is it that _________ will occur? |