Your goal

Your goal is to prepare written assessment tools specifically for each unit or cluster of units. I presume that you are also writing a training and assessment plan, which is normally a separate document. In some cases, tools can be included in the unit description (e.g. essay topics.)

 

What are tools?

Tools are part of a broader grouping called assessment materials. These materials may include:

1. Information for the student, such as a clear explanation of:
• the procedure,
• the expectations
• what (and how much) evidence is required and
• the conditions under which the assessment will be done.
2. Forms used for recording assessments such as:
• checklists for oral tests, observation, or evaluation of work samples
• a profile of acceptable performance measures
• specific questions or activities
• observation checklists
• students' self-assessment materials
• supervisor handbooks and log books
3. Information for the assessor, if it is not apparent in the information to the students. This may be instructions for interpreting evidence (which could also be a moderation tool) to determine competency.

Materials and tools refers to all sorts of things, even essay topics and guidelines for developing report topics.

The terminology is awkward. Some sources refer to tools as synonymous with instruments, or use tools to refer only to recording forms.

 

Good tools

Good tools normally have the following characteristics:

• The task activity is the natural thing to do when performing the skill.
• They don't go on forever; they are are limited. For example, the limited number of questions is enough to collect all necessary evidence. Don't make questionnaires longer than necessary.
• They are re-usable without change. You don't have to create them all over again to assess the next group of students. (In many examination systems, examinations mus be re-written from scratch each time.)
• Tools and instructions are clear enough for other assessors to use them without any further explanation from you..
• They are laid out so that they are easy to read.

 

The rationale

Tools should enable assessors to make concrete judgments and minimize or even eliminate any guesswork. That is, tools should be specific and concrete about what evidence you are trying to collect.

Unit statement

reflected in

Assessment tools

that apply directly to

Real students in concrete situations

The assessment gap is the extent to which assessment is a judgment call.

Unit statements. Very broad, apply to many situations

The assessment gap (guesswork?)

Real students. Concrete, specific situations. Actual evidence to be assessed.

On the left, the elements and performance criteria are meant to be broad enough to be flexible and appropriate to many different situations, and are assumed to be too broad and non-specific to apply clearly to particular situations. On the right is something quite specific: real students in concrete, specific situations producing actual, specific evidence for assessment.

Assessment almost always involves a judgment call, which is supposedly quite risky because it might involve too much guesswork. Training authorities are concerned about the guesswork factor. For example, if a student appeals, can you prove your assessment was correct? How can you show that the assessment was valid, fair, reliable, and flexible?

There is little doubt that too many assessors have made impressionistic judgments that students are competent when they actually aren't. Of course, some assessors are just playing tick-a-box (also known as "tick and flick").

One way to solve the assessment gap problem is to write tools that are much more concrete and specific than the package. They should apply directly to real students in real situations, while also reflecting all package requirements.