|jump to content | main menu | tips on using this site | site map|
Planning the Test
The more time you spend on constructing the test and the accompanying scoring procedure, the less time it will take to score the test. Also, the more time spent in planning, the more confidence you will have that your assessment of the student was done using understandable and defensible criteria.
A course objective is a simple statement of what you expect students to know. An objective has 2 components, behavior and content. Some objectives might be:
The objectives written for a lecture can and should be used as a guide for classroom exams.
Instructional objectives can be written to show different levels of learning.
The most used hierarchy of learning was formulated by Bloom, et.al. The
hierarchy can be used to help formulate objectives. The levels of the
hierarchy, beginning with knowledge, become increasingly complex as we
move up to comprehension, application, analysis, synthesis, and evaluation.
It is assumed that to meet a higher level of learning in a particular
topic, one must first have proficiency at the lower levels.
A link between content and behavior that forms a guide for a test is called the Table of Specifications. Let us suppose that you are teaching a unit on weather. Listed in the first column are the major topics. Across the top of the remaining columns are the types of behavior expected. The remaining job is to fill in the table with the number of items that will be designed at the intersect of the knowledge level and the content area. Note that this Table should be directly related to your objectives.
An objective that could have been used in generating a portion of the above table might be, “The student will be able to select the appropriate symbol for a cold front.” That objective would be reflected in the table by the knowledge of weather symbols for fronts.
From the above table, you can see that air pressure and temperature are not emphasized as much as other topics. This, if course, should be reflected in the teaching of this unit.
A common distinction is made between an objective item (e.g., true-false, multiple choice, etc.) and an essay item. An objective item is called objective because its scoring system is such that almost anyone, when given the key, will get the same score for an item as anyone else. Essay items, however, require expert judges to score them, and even those judges may disagree on the score. Rubrics are schema or criteria that direct the evaluation of student written production and should be developed along with the course objectives. Students are aware of the subjective nature of scoring essays, and sometimes it is feasible to share these rubrics with them so they can direct their studying. In any type of evaluation, it’s important that the score not be seen as biased or capricious.
The type of item you select to measure an objective should be the most direct way to measure the outcome. For example, to determine if a microscope is being focused correctly, you could devise a short answer test or directly observe the students. In this instance, assuming a reasonable class size, observation seems the most direct method of assessment. However, if there is a set of basic facts you wish a student to know, trying to devise an essay topic where all the facts will be mentioned might prove to be impossible. A set of objective test items is more apt to address the objective.
In an objective test of all multiple choice items, it is best to give equal weight to each item in scoring. If you believe one learning objective should be given twice the weight as another, this should be reflected in the number of questions you assign to the objective in your Table of Specifications.
If your test is divided so that some items will take more time to answer than other items (for example, if you have a set of short-answer supply items that will require more work or time than other test items), then the items in that entire set can get more weight than another set of items. These weights should be made clear to the students. Students should always know point values of questions so that they can apportion their testing time.
If you are going to have an essay as part of your assessment, it is strongly recommended that you establish your scoring system before you give your test. An essay test is appropriate when you want to assess the ability of students to organize and apply information. In some content areas you might want to assess writing skills. The degree of freedom given in order to answer a question might be limited or the student may be given a great deal of freedom.
Scoring criteria for assessments should be done in advance. Developing the criteria can lead to a change in the question so that expectations are clearer.
For limited response essays, the instructor should write an example of a response, giving points for the components regarded as important in the answer. For extended response essays, the instructor can use either an analytic scoring rubric or holistic rubric. An analytic scoring method delineates desired characteristics, each characteristic is then rated on a scale (for example, 1 – 5), and descriptions of the scale points within the characteristics is given.
A holistic scoring rubric generates one score. A user of this method must develop a description of the meaning of each of the scaled scores. An outline of the rubric you are going to use should be handed to the students. This will enable them to prepare for the exam.
Performance assessments require a student to demonstrate a skill by actually performing it (e.g., writing a computer program to calculate a mean; carrying out an experiment). Generally these methods of assessment appear authentic. They, too, require a scoring rubric.
Besides scorer unreliability, the principal problems with performance assessments is the amount of time required for the task. Because each task is so time consuming, the number of tasks given is very few. If you have the students perform only a few tasks, it is questionable whether you can generalize to other similar tasks.
|© CET, SFSU 2003||
this is the end of the page.