Planning assessments

When should you plan assessments? You should plan your assessments when you plan the unit for three reasons:

1. Good preparation will make your job much easier. As assessor, you will need to manage people, programs, schedules, and administration responsibilities.
2. You should tell students very early (preferably when you start teaching) how they will be assessed.
3. It will give you a very clear idea of what you are aiming for so you can focus your teaching.

You might need to adjust the assessments as an on-course redirection if you are new to teaching or teaching a new unit that you have never taught and assessed before. Unfortunately, even the best-planned units sometimes hit the brick wall of reality. But that's better than making it up as you go along.

First step: read the units very carefully Unit statements sometimes hide important information, especially in the performance criteria and critical aspects of assessment.

Don't let the (usually) clear, simple layout and boring explanation of the obvious distract you. It is important to get them right. You might want to discuss anything unclear with your supervisor before you go on.

Next, get the big picture.

The right way Getting the whole job done well is the point. Start with an overall picture of what the students must be able to do. The unit title and elements will do the best job of this. Units increasingly also have lists of specific evidence requirements, which is also helpful. Then you can concern yourself with the minutiae of performance criteria etc. etc. etc.

Some expert assessors even suggest that you don't even need to bother students with the performance criteria in the assessment. In theory, they are for assessors to determine how well students perform the elements of competency. (In practice, however, some performance criteria are incorrectly written as stand-alone objectives that don't relate directly to the element. This gives rise to the wrong way to build assessment tools, described below.)

The wrong way Some assessors head straight for the lists of performance criteria. They then try to group them into some new categories to plan assessments. That is, they build upwards from the minutiae. It's like trying to build an elephant out of individual atoms. It ignores the elements, becomes too complex and usually falls in a heap.

Home

Consider language, literacy and numeracy

Consider what language, literacy and numeracy skills are required for the assessment.

You cannot require written work or language skills (e.g. written assignments) that are beyond those specified in the unit or assessment guidelines. This would make the assessment invalid, because it would require something more than the standards. One RTO even uses pictograms to assess occupational health and safety for people with low levels of language and literacy.

As a rule of thumb, do not require any written work from Certificate I students.

You will occasionally find students with a reading disability, or, rarely a writing disability. You will generally need to treat these in terms of allowable adjustments, which we''l get to later.

Home

Consider cost-effectiveness

Examine the cost-effectiveness of your assessment:

• How much lost productivity for the student (and perhaps the supervisor) will there be in the workplace?
• How much time will it take for the assessor to prepare and conduct the assessment?
• What materials will be required for assessment?
• Is the assessment a one-off or can the procedure be used again and again?

This is not so much a factor in many human services programs, but becomes huge when materials and equipment are expensive or when plants must be temporarily closed down.

Home

Make allowable adjustments

Fairness involves making allowable adjustments so that the student is not disadvantaged. Allowable means that they may not compromise program requirements. Most adjustments incur little or no financial outlay, but do take time, effort and thoughtfulness on your part.

Whatever the case, you need to make sure that your assessment will work for your students. It is your responsibility to make any adjustments and confirm them with co-workers and supervisors.

You may have to adjust the assessment for:

1. physical disability involving hearing, vision, voice, mobility
2. intellectual disability
3. a medical condition such as arthritis, epilepsy, diabetes, asthma that is not obvious but may impact on assessment
4. differences in learning progress
5. reading or writing disability
6. denominational or other religious differences
7. cultural background or perceptions
8. age
9. gender.

As a reasonable adjustment, you might need to do one of the following:

1. take into account the student's language, literacy, numeracy requirements
2. give personal support (for example: reader, interpreter, scribe)
3. use special equipment
4. flexible assessment sessions to allow for fatigue or administering of medication
5. change the format of the assessment materials (for example, write them in Braille or the student's first language, use audiotape/videotape)
6. adjust the physical environment
7. revise your proposed assessment methods or tools
8. consider age and gender
9. consider cultural beliefs, traditional practices and religious observances
10. arrange for a member of the community to accompany the student.

Some assessment strategies are not particularly flexible. If the skill is to write a report, then the appropriate assessment strategy is a written report. However, the allowable adjustment might be that the report relates to a topic in which they have some expertise. Some packages specify what is not allowable as an adjustment.

Specific adjustments: Context

The most common kind of allowable adjustment comes from context. For example, an urban youth worker in a church might have different assessment needs to a government youth worker in a country town, though both could be assessed using the same youth worker standards and both might be equally competent.

Other examples are:

• fitting into family schedules
• assessment materials on videotape
• change of venue
• design of shorter assessment sessions to allow for fatigue or medication
• selecting assessment activities that fit their needs or backgrounds
• use of equipment such as a word processor or lifting gear
• allowing special techniques for lefthanders
• using alternative ways of gathering evidence (simulations, role plays, oral assessments)

Specific adjustments: Culture and language

You can make allowable adjustments for culture and language:

• allow work to be done in languages other than English if you have an interpreter or speak that language, and if English is not part of the requirement
• allow other teaching-learning styles
• identify different cultural expectations in learning.

Specific adjustments: Disability

Assessors need to find out whether they need to adapt the assessment to the student's disability. For example:

• A blind person taking a written examination might need a reader, or Braille documents and a Braille writer, a large print version of the paper, or audiotape. He may need either a scribe or a word processor.
• A deaf person may need an Auslan interpreter.
• A dyslexic student may take an interview rather than a written test.
• A sick person may need an attendant carer.
• Students may use a scribe or a word processor if their disability makes them unable to write.

Some kinds of adjustments suit disturbed or intellectually disabled students:

• Increased time (unless efficiency is a requirement)
• Using the same activity for both practice and assessment
• Integrating assessment into regualr class activities to minimise stressfullness.
• Allowing student to take practice assessments before the actual assessment to minimize stressfulness. These may be informal assessments, peer assessments or self-assessments.

Limits of reasonable adjustment for disability

Under the terms of the Act, a reasonable adjustment for a disability may not cause you "unjustifiable hardship". This is defined in terms of the benefit or detriment is likely for the student, the effect of the disability, and the financial cost to you. This means:

• You don’t have to give an adjustment if it won’t make any real difference. This applies to cases where an adjustment provides little benefit for the student, or where lack of an adjustment provides little detriment for the student.
• You must give an adjustment that provides significant benefit for the student, as long as it does not incur unfair expense to you. If the adjustment imposes unfair financial cost, then it is not required (For example, a small organization cannot be expected to hire an extra personal tutor, provide specialized extra equipment, or produce a special edition of texts.)
• If the disability is so serious that the student cannot perform the program requirements within allowable adjustments, you don’t need to make unrealistic allowances. You must give the result "not yet competent." For example, if a blind person shows up for visual arts, you don't have to pass them.

Home

Integrating assessment into classroom schedules

Classroom schedules normally give you limited time to get students to the outcome. Look at all the advantages of integrating assessment into lessons:

1. Assessment is easier to manage if you can use the same activity for practice, formative assessment and final assessment.
2. If you let each stage of a lesson flow naturally into the next, practice progresses easily into the final assessment. Students are better prepared for assessment and less nervous.
3. It is very unlikely that students will be assessed as not yet competent because you have observed their practice and given formative assessments.
4. You can normally get everything done within your time limits. Students have a better chance of being ready for final assessment on time if you notice soon enough what they still need help or extra practice with.

Here’s how a training session can follow these stages:

You can also use the same progression over several lessons or over a whole unit. It is normally a mistake to do a summative assessment in each lesson, and you would have summative assessment towards the end of the unit.

A danger to avoid. If you hold many small assessments over a longer period, you might not put all the bits together. Make sure that it really is summative assessment and that you put all the bits together.

 

Home

 

Assessing higher qualifications

Qualifications at Diploma level and above are driven by knowledge and concepts. Skills usually take a longer time to develop because they take considerable reflection.

Students' RPL assessments are more difficult than lower qualifications for these reasons:

1. Students usually work under minimal supervision and the immediate superior is often at a different location. Getting the supervisor to take a role in assessment is not very helpful.
2. The assessment timeframe is difficult. People at this level of responsibility are decision-makers. But it's a long time between making a major decision and getting results that can be used to evaluate the decision.
3. Assessment requires more expertise. Assessors must be competent subject matter experts, with both theoretical and practical knowledge.
4. The students’ work is conceptual. But knowing a static chunk of knowledge is not as helpful as being able to find and evaluate knowledge and to generate new knowledge.
5. Students work in various scenarios with changing variables. Just knowing how to do something isn’t enough; they need to know how to do it differently in different situations. If you ask them how to do something, they’ll begin with: "It depends on … " Sometimes they must plan for situations that can only be envisaged or forecasted.

 

Interviews

Interviews based on questions are especially appropriate. Most of the questions below are open-ended questions, that is, they cannot be answered with "Yes" or "No". Even when students give an answer, you usually need to ask the reasons why.

1. Use questions to get information.
• How often do you review progress in that department?
• What strategies did you try?
2. Ask for an analysis. You can ask the student to draw conclusions or to identify causes, variables, or significant factors.
• What are the main elements in your program?
• What are the main steps in how you did that?
• Why do some succeed and some fail?
• Why did those difficulties arise?
• What conclusions did you draw?
• What are the significant factors in that approach?
• What variables affect it?
• "What kind of things does that depend on?"
3. Use questions to get students to defend their arguments.
• Why did you do it that way?
• What evidence supports that conclusion?
• How would your respond to the idea that ... ?
4. Ask the student to anticipate trends and to plan ahead
• What needs to be done to make that idea work?
• How will that plan work in the market place?
5. Ask students to choose priorities and defend their choice.
• Of the three problems, which one will you try to solve first?
• Why did you spend more time on that option?
6. Hypothetical questions ask "What would happen if …?"
• What would have happened if John had resigned at that time?
• How would your colleagues have reacted to a reduced share price at that stage?
• What if you had a funding cut of 10%?
7. Explore implications
• What are the implications of that kind of policy change?
• How would your members react to those recommendations? How would you answer them?
8. Make generalizations
• Based on your experience, what are the major risks?
• What are the main reasons for that kind of strategy?

 

Home

 

Using third parties to gather evidence

With thanks to Russell Docking

Using local supervisors to gather evidence can cover hundreds or thousands or hours of work, more than an off-site RTO staff member could gather. They are not qualified assessors, so they are not actually making the assessment judgment, just gathering or providing evidence.

 

Orientation

To start, give orientation to supervisors to tell them what to expect.

1. Give supervisors a basic idea of Competency Based Assessment. CBA is fairly easy to understand: "We need to know if they have the skills to do the job properly. If they can't actually do it, it doesn't count.") This is usually fairly easy except when they have ingrained preferences to higher education ("They have to understand it. Doing the job is not so important.")
2. Give supervisors specific guidelines. They will need to know what you expect and when. Some will need deadlines. Any information needs to be in normal workplace English.
3. Do some moderation exercises. For example, you could show them examples of satisfactory and unsatisfactory performance.
4. Talk about records:
1. Provide record books that are ready to be filled in.
2. The records should be contemporaneous, done at the time of observation, not filled in from memory just before you show up.
3. The particular skills or tasks should be signed off, not the individual criteria.
4. Supervisors should write comments on what they saw, what the student did well, what student found difficult, etc.
5. They should be signed and dated.
6. Show them examples of record books filled in well and those filled in badly.
7. Anticipate that some supervisors will probably be surprised at the idea of good performance but poor records, and poor performance but good records. If the student's performance is poor, you'll need records that would hold up on appeal. There are four basic possibilities, and you might want to have examples of each:
• Good performance, good records
• Good performance, poor records
• Poor performance, good records
• Poor performance, poor records.

 

Ongoing Support

You will also need to give supervisors ongoing advice and support. Many suffer in silence if they get stuck, waste lots of time, and either over-record or under-record.

Plan to visit or telephone periodically. Ask how they're going and for any questions they might have. They might also unsure and lack confidence, and simply need to be old that they are doing it right. If you visit, see what they do.

As part of your moderation, you might want to have a co-assessment session during a visit on the job.

 

Other evidence

You might also need to discuss student performance with the supervisor, which you would record as an interview for extra evidence if the supervisor is averse to writing. The workplace will probably generate other kinds of naturally-occurring evidence such as job descriptions, log books, reports, products, and work evaluations. Obviously, you'll want to put it into a portfolio.

 

Home

 

Information that students might need

You need to explain clearly to students a considerable amount of information on how assessment will be done. It must be provided before you start assessing, and often before you start teaching. It can be given in different ways:

• Some information is included in your college’s and general student information and policies.
• You can give some information to a whole class.
• You need to give some kinds of information to the individual student.

The kinds of information are as follows:

1. The purpose of the assessment, usually by mentioning the units or a qualification (i.e. the level and name) he/she would get upon satisfactory performance.
2. The date, time and place of assessments. If these are negotiable, then you need to be sure that both parties are in agreement.
3. Exactly what will be assessed.
4. How it will be assessed (i.e. the methods/modes, tools, and procedure, and the kind of evidence that will be gathered.)
5. The criteria against which they will be assessed (which may be implied in the teaching.)
6. Particular evidence requirements
7. Any relevant instructions and regulations.
8. Legislation and other regulatory requirements that affects the student.
9. Any standards for higher levels of competence, (e.g. graded assessment).
10. Explain any legal or ethical responsibilities to the students.
11. What happens if the student is considered competent or not yet competent.
12. How many chances they have to be assessed (usually another attempt).
13. If the assessment is unfair in some way, they can appeal.

 

Other information

When you start a new unit, you'll probably need to orally explain the outcomes to students. They often don't understand a written statement of outcomes because they haven’t studied them yet.

You might be well advised to mention several other matters:

1. Any formative assessments done as part of learning experience
2. Check that the student is correctly enrolled; it is seldom necessary but incorrect enrolments have prevented students from receiving qualifications.
3. Prevent misunderstandings about payment of fees because fees are often linked to enrolment.

With the possible exception of a campus group, you might need to include more information in the assessment plan. This would involve cases such as:

• several on-the-job assessments
• getting permission from a supervisor
• scheduling other people
• scheduling equipment
• documents that are difficult to get or should include particular wording
• special safety issues
• the list of assessment criteria is quite lengthy and complicated.

Your RTO should explain the following to students when they are admitted, but you might find them within your role as instructor or assessor:

• The VET Sector
• The assessor’s qualifications
• That the student has the right to view his/her file
• Workplace assessment policies
• Information on the Registered Training Organisation
• The appeals process

 

I'm quite sure that people who learn "class-room knowledge," and people who learn on the job have developed very different structures for their cognitive knowledge. (It would be quite difficult to prove and would be a good dissertation topic for someone doing a PhD in educational psychology.) In my experience, classroom students find some theory questions answerable if they have done their study. However, competent professionals who learnt on the job often perceive them to be too difficult or even unanswerable. They know what to do and why, but don't articulate theory in the same way a textbook does. This is still true when the practitioner has mastered the theory. The reverse is also true; the same questions rephrased for competent professionals are perceived to not make sense for classroom students. The first implication is that it is easy to fail an RPL student by using "class-room knowledge" kinds of questions, but the exact same student can pass with flying colors if given equivalent "professional practice" questions, for example, "Given situation X, what would you do and why?" The second implication is that students who learn in a campus situation are full of "textbook knowledge." They have to re-think it all to make sense of it in a practicum situation. There is plenty of space for it to be lost in translation. Besides cognitive factors, context plays a role because location and people trigger responses, and people haven't plugged into the triggers yet.