A failure of imagination

A discussion of assessors' impressions on vocational education, with Amy Bolezny, Phillip Rutherford, and Ross Woods

Many assessments fail, not just for technical reasons, but because writers of competency standards and assessment tools forget to imagine what the job actually requires. They fail to see the essential link between the competency standard and the reality of what workers do on the job.

The fatal flaw

Many writers of assessments at higher certificate and Diploma levels seem to have been indoctrinated in the belief that they must set academic-style assessment tasks. Admittedly, essays have a place in an academic environment to promote research and critical thinking, and paper-based learning can be good or even unavoidable in vocational training and assessment. Yet these approaches should not be the default position.

This misplaced trust in academic-style assessment tasks arises from various causes:

These fatal flaws prevent otherwise good assessors from translating what workers actually do on the job into assessment tasks that reflect best practice.

“Naturally occurring” evidence

Compared to disembodied academic tasks, it is more effective to use “naturally occurring” evidence, that is, the evidence of skills and knowledge that occurs naturally in the workplace by actually doing the job. It is neither forced nor manufactured, and might not even require the student to especially demonstrate skills for the purpose of assessment.

Here’s an example from management:

Naturally-occurring evidence is the purest form of evidence possible. However, it is the most difficult form of evidence to gather if the college is an offsite, independent body. For example, people often exaggerate when asked to provide testimony to someone's performance, or they fail to incorporate the full action along with the context within which it was performed. Saying that "Such-and-such gave good customer service" doesn't quite explain what was done and how.

It is much easier to collect naturally-occurring evidence if assessment tools are workplace-centered, supervisors become co-assessors, and the assessment system is part of the organization’s performance management system. By doing so, assessors continually assess the progress of individual and group competence towards organizational objectives as well as helping individual staff members pick up qualifications along the way.

When training is directly linked to the achievement of organizational goals, such as profit or change, organizations are more likely to invest in training because it can see direct returns from their investment.

Conclusion

If we train our fledgling assessors and colleagues to use their imaginations, vocational colleges can better exploit their opportunities to have a greater impact on their clients, and fewer would fail audits through inferior assessment practices.