 |
|
 |
|
 |
|
 |
|
 |
Printer-friendly page
Planning the Test
| This Page Includes |
|---|
Introduction
Planning the Test
Developing Course Objectives
Activity
Table of Specifications
Objective & Essay Test Items |
Planning the Test
The more time you spend on constructing the test and the accompanying scoring procedure, the less time it will take to score the test. Also, the more time spent in planning, the more confidence you will have that your assessment of the student was done using understandable and defensible criteria.
Developing Course Objectives
A course objective is a simple statement of what you expect students to know. An objective has 2 components, behavior and content. Some objectives might be:
| Behavior | Content |
|---|---|
| Balance | a chemical equation |
| Explain | the role of food in classical conditioning |
| Devise | a double-blind study |
| Name | 3 elements from the halogen group |
The objectives written for a lecture can and should be used as a guide for classroom exams.
Instructional objectives can be written to show different levels of learning.
The most used hierarchy of learning was formulated by Bloom, et.al. The
hierarchy can be used to help formulate objectives. The levels of the
hierarchy, beginning with knowledge, become increasingly complex as we
move up to comprehension, application, analysis, synthesis, and evaluation.
It is assumed that to meet a higher level of learning in a particular
topic, one must first have proficiency at the lower levels.
Knowledge is knowing specific facts, principles, etc.
Comprehension is the ability to explain a point.
Application is using previously known facts to solve a problem.
Analysis is the ability to break a product apart into its requisite elements or logical components.
Synthesis is the ability to create something.
Evaluation is the ability to judge quality.A helpful list of words that are used for different types of objectives follows
| Knowledge | Comprehension | Application | Analysis | Synthesis | Evaluation |
|---|---|---|---|---|---|
| define identify indicate know label list memorize name recall record relate repeat select underline |
classify describe discuss explain express identify locate paraphrase recognize report restate review suggest summarize tell translate |
apply compute construct demonstrate dramatize employ give examples illustrate interpret investigate operate organize practice predict schedule shop sketch translate use |
analyze appraise calculate categorize compare contrast criticize debate determine diagram differentiate distinguish examine experiment inspect inventory question relate solve |
arrange assemble collect compose construct create design formulate manage organize perform plan prepare produce propose set-up |
appraise assess choose compare contrast decide estimate evaluate grade judge measure rate revise score select value |
 |
|
Table of Specifications
A link between content and behavior that forms a guide for a test is called the Table of Specifications. Let us suppose that you are teaching a unit on weather. Listed in the first column are the major topics. Across the top of the remaining columns are the types of behavior expected. The remaining job is to fill in the table with the number of items that will be designed at the intersect of the knowledge level and the content area. Note that this Table should be directly related to your objectives.
Sample of Table of Specifications
| Content | Knows Basic Terms |
Knows Weather Symbols |
Knows Specific Facts |
Understands Influence on Weather Information |
Interprets Weather Maps |
Number of Items |
|---|---|---|---|---|---|---|
| Air Pressure | 1 |
1 |
1 |
3 |
3 |
9 |
| Wind | 1 |
1 |
1 |
10 |
2 |
15 |
| Temperature | 1 |
1 |
1 |
4 |
2 |
9 |
| Humidity & Precipitation | 1 |
1 |
1 |
7 |
5 |
15 |
| Clouds | 2 |
2 |
2 |
6 |
- |
12 |
| Fronts | 1 |
1 |
5 |
5 |
5 |
17 |
An objective that could have been used in generating a portion of the above table might be, “The student will be able to select the appropriate symbol for a cold front.” That objective would be reflected in the table by the knowledge of weather symbols for fronts.
From the above table, you can see that air pressure and temperature are not emphasized as much as other topics. This, if course, should be reflected in the teaching of this unit.
Helpful tips
- It is typically not necessary to differentiate between the two levels
of knowledge and comprehension and among the three levels of analysis,
synthesis, and evaluation.
- Typically knowledge and comprehension are regarded as somewhat lower
level items (where the knowledge is more passive in nature).
- Analysis, synthesis, and evaluation (where the examinee usually has
to generate responses or evaluate possible solutions) are considered
as higher level items. Your time can be better spent selecting or generating
useful objectives or test items.
- Also, in introductory courses, the bulk of learning and instruction
will occur at the level of application and below.
- In order for higher level learning to occur, the more basic levels
must be well learned.
- Higher levels may not be able to be exhibited until you reach a higher course level.
|
|
Objective and Essay Test Items
A common distinction is made between an objective item (e.g., true-false, multiple choice, etc.) and an essay item. An objective item is called objective because its scoring system is such that almost anyone, when given the key, will get the same score for an item as anyone else. Essay items, however, require expert judges to score them, and even those judges may disagree on the score. Rubrics are schema or criteria that direct the evaluation of student written production and should be developed along with the course objectives. Students are aware of the subjective nature of scoring essays, and sometimes it is feasible to share these rubrics with them so they can direct their studying. In any type of evaluation, it’s important that the score not be seen as biased or capricious.
The type of item you select to measure an objective should be the most direct way to measure the outcome. For example, to determine if a microscope is being focused correctly, you could devise a short answer test or directly observe the students. In this instance, assuming a reasonable class size, observation seems the most direct method of assessment. However, if there is a set of basic facts you wish a student to know, trying to devise an essay topic where all the facts will be mentioned might prove to be impossible. A set of objective test items is more apt to address the objective.
In an objective test of all multiple choice items, it is best to give equal weight to each item in scoring. If you believe one learning objective should be given twice the weight as another, this should be reflected in the number of questions you assign to the objective in your Table of Specifications.
If your test is divided so that some items will take more time to answer than other items (for example, if you have a set of short-answer supply items that will require more work or time than other test items), then the items in that entire set can get more weight than another set of items. These weights should be made clear to the students. Students should always know point values of questions so that they can apportion their testing time.
If you are going to have an essay as part of your assessment, it is strongly recommended that you establish your scoring system before you give your test. An essay test is appropriate when you want to assess the ability of students to organize and apply information. In some content areas you might want to assess writing skills. The degree of freedom given in order to answer a question might be limited or the student may be given a great deal of freedom.
|
"Briefly summarize the experiment done to determine ... " By comparison, the following question would elicit a more extended response: "Compare the two major theories of alcoholism paying special
attention to the philosophical background of the theories, methods of
treatment, and evidence supporting each. Which theory do you believe
is best supported by the evidence? What aspects of the research brought
you to this conclusion?" |
Scoring criteria for assessments should be done in advance. Developing the criteria can lead to a change in the question so that expectations are clearer.
For limited response essays, the instructor should write an example of a response, giving points for the components regarded as important in the answer. For extended response essays, the instructor can use either an analytic scoring rubric or holistic rubric. An analytic scoring method delineates desired characteristics, each characteristic is then rated on a scale (for example, 1 – 5), and descriptions of the scale points within the characteristics is given.
A holistic scoring rubric generates one score. A user of this method must develop a description of the meaning of each of the scaled scores. An outline of the rubric you are going to use should be handed to the students. This will enable them to prepare for the exam.
Performance assessments require a student to demonstrate a skill by actually performing it (e.g., writing a computer program to calculate a mean; carrying out an experiment). Generally these methods of assessment appear authentic. They, too, require a scoring rubric.
Besides scorer unreliability, the principal problems with performance assessments is the amount of time required for the task. Because each task is so time consuming, the number of tasks given is very few. If you have the students perform only a few tasks, it is questionable whether you can generalize to other similar tasks.