Giving assessment

Your goal in assessment

Your goal in assessment is to gather evidence and make an assessment decision. It's doing what you planned to do. Previous ebooks told you how to plan the assessment. That included choosing the competency standards and other relevant benchmarks, and interpreting them so they would work in your situation. If you wrote your own assessment tools, you then chose the activities, designed the assessment (taking into account students' needs), and planned any reasonable adjustments.

It’s now time to put all that planning into practice. Gathering evidence is basically carrying out the plan. Use assessment tools for gathering evidence or give them to students so that they can gather evidence themselves (e.g. for a portfolio). Ensure that evidence is organized in a suitable format so that you can assess it. Keep an eye on students during the assessment; they may need help if they get stuck.

Relating to your students

Nobody enjoys being assessed. Students are often nervous, fearful, and in need encouragement. Your attitude affects student performance. If they feel that you have created a friendly, supportive environment, they will perform more satisfactorily than if you appear to create a hostile environment.

Your first step is to establish a working relationship with the student. Of course, you’ll need to maintain it throughout the assessment.

Your attitude is important. You will need to guide and support students and give them encouragement. You will need to be sensitive to any issues arising, including your own role in the relationship. Be humble enough to accept and utilize feedback.

If you observe students a little, you will figure out the best kind of interpersonal approach you need. You will need to develop a professional relationship, but you will still need to be sensitive to students’ individual differences. This is especially the case for members of potentially marginalized groups.

Tip 1If you do on-job training, you can usually delay the assessment until students are up to speed and ready. Then almost all your students will pass well. | Tip 2In vocational education, the best way to start an assessment is to get the student to show you around their workplace and tell them what they do. You can ask them anything they don’t explain. They’re quite comfortable on their home turf and will often tell you most of the things you want to know. Then when you come to the formal assessment, you be able to skip anything they’ve already explained in full. | Tip 3Students are normally nervous before any assessment, so it’s essential that you put them at ease beforehand.

A two-way discussion

You will also need to discuss the assessment with students and give them full opportunity to ask questions, even if they are shy or reticent.

Your discussion with the student should be two-way, with you being an active listener who asks questions to clarify what was said and confirm that you both understand. Watch out for non-verbal messages as well as verbal messages; make sure you interpret them accurately.

You need an appropriate communication style for the specific context. For example, it should mirror the language used in the context, so that students understand it and feel comfortable with you.

Establish ways to encourage communication and feedback between you and the student; make sure you can both communicate anything necessary. Be careful to avoid monologue explanations. Let the student ask questions and clarify anything that he/she is unsure about. If you and the student have an agreement to be honest with each other, then it can benefit you both:

• The student has the freedom to tell you if they are nervous or if they normally do the assessed task in a very different way.
• You have the freedom to give praise for good work and the right to mention anything unsatisfactory.

Ask the student about his/her preferences, needs and expectations, and address any inclusivity issues and other potential problems. You may negotiate to achieve an assessment approach that works for both you and the student.

You need to make sure the assessment goes smoothly, saves time and money, and gives the student a fair chance of success. Some assessment authorities are of the opinion that fairness involves fully informing students of the assessment criteria before they are assessed; in that view, it is not enough to tell them verbally or to put a copy on a notice board. Others would want a general picture of expectations, but would not require students to understand the details of the criteria. Either way, it is unfair to assess students without first telling them what is expected.

What you must tell students

Make sure that students are informed of expectations before the assessment. In postsecondary education it is normal practice to provide students with written information on the assessment at the beginning of the term or semester, usually in the unit description. Be aware that students may have lost written information given to them at the beginning of the unit.

During the whole process, you will need to:

• Give the student clear and concrete presentations of options
• Explain the purpose of assessment to the student
• Explain the assessment procedure and methods proposed
• Give clear and precise instructions
• Give appropriate constructive feedback to student
• Discuss the assessment outcome with student

Take time to explain to students any factors affecting the assessment. Your admissions officer might have handled some of these, but you should explain anything still unclear. The point is that the students should go into the assessment knowing what to expect so that there are no nasty, unfair surprises.

What you might not be able to tell students

In some cases, the student is not permitted to know some things about the assessment. For example:

• Many written tests do not work if students are told the questions beforehand. But they should be told what will be covered in the test and the kind of answers expected, such as short or long answer, multiple choice, etc.
• A shop assistant may be assessed through a mystery shopper. A stranger comes into the shop to buy something, but the student does not know whether it is a real customer or an assessor. In this case, inform the student that they will be assessed through mystery shoppers.

Check for other factors that might affect the assessment

The list of assessment factors is quite long, and many might not apply in particular situations. Factors affecting the assessment may be:

• relevant competency standards in use
• the environment in which the assessment will be done
• opportunities for gathering evidence in a number of situations
• who will make the assessment
• relationship between units and student’s workplace
• auspicing and partnership arrangements
• how long the assessment will take
• costs and fees
• relevant quality assurance procedures
• holistic assessment
• privacy and confidentiality
• a concrete understanding of the assessment process
• further opportunities for re-assessment
• your RTO's appeals processes.

The assessment decision

You must make an assessment decision that is in line with the standards and procedures that you are using and the evidence that you have gathered. Remember your limitations; you can ask an experienced assessor for help if necessary. It is a judgement call, but the closer the fit between evidence and assessment criteria, the easier it will be.

The dominant factors in making assessment decisions are the assessors' own assumptions and biases.

Making an assessment judgement can be defined as a two-step process:

• First, look at the quality and breadth of evidence. Does it meet the general standards of assessment? Do you have enough evidence to make a decision and justify it?
• Second, based on the evidence provided is the student competent or not yet competent?

If you are using a graded assessment system, it is then another step after that.

Handle your biases

Assessors are people and have unconscious biases. It's not just everyone else either. It includes you and me.

The best antidotes are:

• become conscious of them so you can monitor yourself
• ensure students are ready for assessment
• moderate your assessments with other assessors.

If you are assessing students whom you taught, formal assessment usually doesn’t bring many surprises. You should have a very good idea of students’ learning effectiveness before you do any formal assessment.

• In any kind of teaching, your lesson should include a way of knowing whether students achieved the purpose of each lesson. Even with bigger groups, you should have a good idea of whether students have learnt what you taught them.
• If you have taught an item before, you probably have a good idea of which points students most frequently find difficulty.
• If you are a classroom teacher in an elementary school, you probably have a relatively small group of students, have more time with them, and spend more time monitoring their work. After the instruction section of most lessons, you probably spend time walking around class observing student work, and answering questions.

On the other hand, beware of assumptions based on past experience.

• Your past experience of particular students might bias your assessment. For example, you expect a strong student to do well so it is easy to give a higher grade than they actually earn.
• Keep alert to assessment results that oppose your experience. It might indicate that lessons weren’t as effective as you thought. It might also indicate that the assessment was flawed in some way.

Professionalism in assessment requires that you realize when you are biased and adjust your assessment approach. Here are some of the main biases that unconsciously weaken assessment:

1. Assessors tend have an idea of average. This results in two similar tendencies:
• Assessors could make average performance the pass mark. (This is actually norm referencing. The middle 50% pass, the top 25% get distinctions, and the bottom 25% fail.) For example, if you assess a large group, you will unconsciously be tough at first and become more lenient as you realise what is average; or normal.
• You presume the student is average unless proven otherwise. You might base your assessment result on a view that the whole group or class is ordinary, exceptional or struggling.
   
2. Assessors tend to let a strong performance in one area compensate for inadequate performance in another area. This is especially common among those with a background in teaching Higher Ed. If a student is not yet competent on some elements, he/she cannot compensate for it by doing very well on others. The student is clearly not yet competent on some elements and they must pass all elements to be considered competent for the unit.
    
3. Assessors can tend to presume the student to be competent if he/she:
• is confident, articulate, or personally attractive
• gave a good first impression
• has a track record of good performance
• is competent in other related skills, or
• holds similar opinions to you, the assessor. (This assumes: I’m good so I’m the benchmark for others.)
   
4. Assessors can tend to presume the student to be not yet competent if he/she:
• lacks confidence
• is inarticulate
• is unattractive
• has a track record of poor performance
• is not yet competent in other related skills, or
• holds different opinions from the assessor.
   
5. Assessors can tend to make inferences on critical aspects of performance because they failed to observe them.
    
6. Some assessors fail to make written records at the time, and later write down what they remember.
    
7. Assessors can tend to be sympathetic to the student’s dilemma. It is difficult to give a not yet competent; result to a student who has financially sacrificed to take the course or whose job (or future job) depends on a competent outcome.
    
8. Assessors can face their own dilemmas. Your college might get a bad name if you fail too many of your students and you might get fewer clients.

Many of these can also work in the negative, biasing you to presume that the student is not yet competent. For example, you might presume they’re not yet competent at one thing because you’ve seen that they’re not yet competent at something else.

Decision-making strategies

A great deal of the assessment problem is elimination of doubt, the fuzzy area where the decision could go either way. In the doubt gap, no decision is really defensible:

You are caught between two options. First, if you judge the student competent, could you justify your decision? Second, if you judge the student not yet competent and the student appealed, could you justify your decision? Either way, would someone else independently reach the same conclusion? How would you answer a supervisor or auditor?

You can seldom be absolutely sure. It’s always possible to ask for more evidence and then face the problem of over-assessing. Besides, assessment criteria at the higher levels are usually abstractions, which can always be reinterpreted. We only need to be sufficiently sure based on sufficient evidence. So how much is sufficient? Enough for someone else to independently draw the same conclusion from the same evidence.

In a good assessment decision, you can show which side of the gap the student is on, because there is enough evidence to minimize or even eliminate doubt. As we saw, good assessment tools go a long way to put you in this situation.

There is more than one way to view the an assessment decision process:

1. You cannot assume they’re competent unless proven otherwise. This assumption might unreasonably help the student who doesn’t have enough evidence. It’s easy enough to do. (I know it’s a bit thin, but they showed up and tried hard.)
2. You usually can't simply weigh up pros and cons. How do you handle the doubt gap?
3. You can assume they’re not yet competent but give them fair and reasonable opportunity to prove otherwise. This is a good solution. It is the student’s responsibility to produce enough evidence to demonstrate competence. In other words, you accumulate enough evidence to make a reliable judgement. It is the assessor’s responsibility to provide tools the enable students to do so.

In other words, if you conclude that you have insufficient evidence to make a decision, you must give a Not yet competent decision. But you'd normally give the student a chance to supply more evidence.

Risks

You can make your assessment decisions more reliable by calculating the risk of making a wrong decision. Most assessments have some level of risk, especially RPL where evidence comes from a variety of sources. Besides, students with dodgycredentials really need RPL because the assessment process will hopefully result in a nationally recognised qualification.

If you assess a person as competent when in fact they are not, there may be legal consequences. If you assess them as not yet competent when in fact they are, it may adversely affect your RTO’s reputation and result in a messy appeal.

Risk levels

Very low risk. The risks are very low if you taught the students yourself and monitored their learning progress.

Low risk. The prima facie risks are low when, for example:

1. The student gives you primary documents (reports, theses, etc)
2. The student satisfactorily answers all reasonable questions about what they have done
3. The student works under a well-documented accountability system
4. The student has another qualification that is relevant to the units you are assessing
5. You can verify the reference through a phone call or email based on information not received through the student (e.g. phone book, website, accreditor, etc.)
6. The student has been observed by a range of persons over a period of time
7. You already trust the expertise and judgement of the people giving references
8. The student has been independently assessed by industry or statutory bodies according to standards that you know to be rigorous. (And it’s not a bad idea to keep a copy of the standards and/or procedures in the student’s evidence file.)

Higher risks. The opposite is also true. Risks are higher when:

1. The student has a weak or poorly documented accountability system
2. You suspect that the student is dishonest
3. References are not convincing (making them nearly worthless):
   • References only refer to character and omit competence
   • You do not know the people or organizations giving references
   • References from people in organizations are given personally rather than through the organization
   • References cannot be verified in any way
4. The student has no related qualifications or assessment records
5. The items of evidence are inconsistent with each other
6. You have observed the student only the minimum number of times
7. The students’ actions are inconsistent with the competencies to be assessed. (For example, a student taking the assessment units offers evidence that is not authentic, not current, or insufficient.)

And risk levels can change ...

To make matters more complex, the risk status might change during the assessment. For example, you might doubt a reference from an obscure organization. Yet the organization might be excellent and highly credible, just that it was previously unknown to you.

Change in risk status can also go the other way. A reference appears very useful, but then you find that the person giving the reference has a possible conflict of interest. You might still count the reference as evidence, but only to corroborate lower-risk evidence items.

As another example, a student might offer a license from an international organization as evidence of competence. So far, it appears to be low risk. But then you find out that the international organization is one person acting alone with no formal recognition other than a registered business name, and the procedure for getting the license is very weak. That is, the license is worthless as evidence of competence. The assessment is also higher risk because the student has offered such flimsy evidence.

In case of a higher risk assessment, you may simply need to get better evidence to minimize risk. In fact, people who know that their current credentials are weak or poorly recognized are those who most need a successful RPL assessment to gain a recognized qualification.

What to do

Verify evidence. It is normal good practice to verify evidence, and necessary if there is any reason to doubt it. Besides an email, phone call or visit, the organization’s website may be helpful.

Get more or better evidence. If the evidence is too risky to clearly demonstrate that the student is competent, you might need to decrease the risk by collecting more evidence. Otherwise you are forced to state clearly why the evidence is high risk (e.g. reference cannot be verified) and give a not yet competent decision.

The sudden death factor

A few assessments have a sudden death factor. If students get something essential wrong, they must be assessed as Not yet competent, no matter how good they do everything else in the assessment. This is different from a minor error made by a competent student who still has room for improvement.

Here are several examples:

1. If you break the law only once in a driving test, you've failed the test.
2. If a student's essay is built on a major contradiction or fallacy, you can't give it a passing grade. In this case, a major contradiction or fallacy is different from a minor error that does not affect the conclusion.
3. If a student acts dangerously in a real workplace during an assessment, he/she has already failed the assessment. You may even need to stop it. In this last case, acts dangerously is different from having a minor risk that does not seriously compromise safety.

Give students more than one chance

In many kinds of skills, it's only fair to give students a second chance if they didn't demonstrate it the first time. It's basically unfair to write off students who fail on their first attempt, especially if they're very nervous.

This is not a problem in straightforward demonstrations of skill, where classroom scheduling might not be a factor. Supplementary examinations might be possible in other cases.

However, you are not obliged to give students more than one chance in all situations. For example:

• A student might assemble and submit a portfolio for assessment. You could allow resubmission to cover inadequacies, but you do not need to provide a full second assessment for free.
• You would not normally give a second chance for seriously dangerous, unethical or illegal behavior. In these cases, the student might also be removed from the program or dismissed from the workplace.

Question: What if a student is assessed as not yet competent on the second assessment attempt? Do I then give him/her another chance? How long can this process go on for?

Answer: The student is entitled to a second chance. Even if he/she doesn't get the whole qualification, the college must issue a transcript for any units in which you assess him/her as competent. If you wish, you could give the student a third chance, but you don't have to and he/she is not really entitled to it. You are quite within your rights to say, This was your second assessment of these units, and you weren't successful. You are welcome to try again, but you'll need to apply and pay again for them.uot;

Give encouragement and feedback

Students are normally very keen to know how the assessment went, and apprehensive about the result. Their first question is: Did I pass? If they gain a competent result, they might be so relieved that they don’t care about your feedback at the time. If you write it down, they can come back to it later.

Feedback to students can include lots of things:

• confirming results
• telling them that they did well
• telling them the reason for the result (especially graded assessment)
• discussing plans
• confirming the kind of gap training needed
• identifying further evidence to be provided
• suggesting improvements in gathering and presenting evidence
• providing the student with opportunities for further development
• giving guidance on future options.

A.S.A.P.

Give feedback as soon as possible after the assessment. It will have more effect and students will remember the correction rather than any mistakes they might have made. The longer you delay, the less notice the student takes.

Otherwise, the timing varies greatly depending on the assessment mode. Assessors can give students immediate feedback on an individual oral examination, but a large pile of written assessments takes days or even weeks to assess.

Be positive

Feedback should be clear, constructive, and helpful. Be polite and positive, and comment on things well done. Give more positive than negative comments so people feel encouraged rather than crushed. It's good advice to start with positive comments, then give negative comments and conclude with positive comments.

Where possible, this is the most appropriate stage to give the student guidance on further opportunities for training. If there are gaps in the student's abilities, this is a good time to talk to the student about them.

Your task is more difficult if the student has done an assessment task unsatisfactorily. They might be despondent, disappointed, defensive, or angry about a not yet competent result. A negative result may have serious employment ramifications for them (e.g. they lose a promotion or lose their job). If you have bad news, it is your job to give it, and you do have to tell the truth.

• Be clear and specific on the deficiency and remind them of what was expected. This will also make appeals less likely.
• Tell them what they will need to repeat.
• Help them plan to fix any weaknesses.
• Advise them to choose future electives in their areas of strength.

Put your comments in writing

Verbal feedback is usually necessary, but it's good practice to write it down too. Giving feedback is often an accreditation a requirement and you may need to prove that you have done so in an audit.

Get their feedback on how you went

As the assessor, you should get feedback from the student on how they thought the assessment went.

Record and report assessment results

Use the forms that you developed in the planning process. Assessment records are obviously extremely important and must be made at the time of assessment; they cannot be done by memory later on.

Many colleges have their own procedures for records and deadlines for the submissions. Where the law affects assessment (e.g. some kinds of licensing) compliance with legislation should be built into your college's procedures.

You need to:

1. give your supervisor feedback on the positive and negative features of the assessment, and perhpas suggestions for revision.
2. tell your supervisor if the student disputed any result,
3. appropriately record details and submit the written report of results for filing,
4. see that all records are stored securely in a safe place. In some cases, you will file them yourself. In other cases administrative personnel will have that responsibility, especially if results are also kept on a computer database.
5. provide students with a written report. (An informal oral report straight after the assessment is good practice but not usually adequate by itself.) There are several reasons, one of which is simple fairness. The other is that the student might want to appeal against a result, and your college must have an appeals process in place.
6. maintain the confidentiality of assessment results.

Starting considerations

Here are three good reasons to plan your assessments when you plan the unit:

1. Good preparation will make your job much easier. As assessor, you will need to manage people, programs, schedules, and administration responsibilities.
2. You should tell students very early (preferably when you start teaching) how they will be assessed.
3. It will give you a very clear idea of what you are aiming for so you can focus your teaching.

Planning is better than making it up as you go along. You might need to adjust the assessments as an on-course redirection if you are new to teaching or teaching a new unit that you have never taught and assessed before. Unfortunately, even the best-planned units sometimes hit the brick wall of reality. However, it is more efficient to change a plan than to have no plan at all.

Step 1: Read the competency statements very carefully.

They sometimes hide important information, especially in the performance criteria and other details. Don't let the (usually) clear, simple layout and boring explanation of the obvious distract you. It is important to get them right. You might want to discuss anything unclear with your supervisor before you go on.

Step 2: Next, get the big picture.

In vocation and professional education, the the point is to get the whole job done well. Start with an overall picture of what the students must be able to do. Then you can concern yourself with the minutiae of performance criteria etc. Some expert assessors even suggest that you don't even need to bother students with the performance criteria in the assessment. In theory, they are for assessors to determine how well students perform the elements of competency. (In practice, however, some performance criteria are incorrectly written as stand-alone objectives. This gives rise to the wrong way to build assessment tools, described below.)

The wrong way. Some assessors head straight for the lists of performance criteria. They then try to group them into some new categories to plan assessments. That is, they build upwards from the minutiae. It's like trying to build an elephant out of individual atoms. It ignores the elements, becomes too complex and usually falls in a heap.

Step 3: Consider language, literacy and numeracy

Consider what language, literacy and numeracy skills are required for the assessment.

You cannot require written work or language skills (e.g. written assignments) that are beyond those specified in the competency guidelines. This would make the assessment invalid, because it would require something more than the standards. One vocational college even used pictograms to assess occupational health and safety for people with low levels of language and literacy.

You will occasionally find students with a reading disability, or, rarely a writing disability. You will generally need to treat these in terms of allowable adjustments, which we''l get to later.

Step 4: Consider cost-effectiveness

Examine the cost-effectiveness of your assessment:

• How much lost productivity for the student (and perhaps the supervisor) will there be in the workplace?
• How much time will it take for the assessor to prepare and conduct the assessment?
• What materials will be required for assessment?
• Is the assessment a one-off or can the procedure be used again and again?

This is not so much a factor in many human services programs, but becomes huge when materials and equipment are expensive or when plants must be temporarily closed down.

Allowable adjustments

Fairness involves making allowable adjustments so that the student is not disadvantaged. Allowable means that they may not compromise program requirements. Most adjustments incur little or no financial outlay, but do take time, effort and thoughtfulness on your part.

Whatever the case, you need to make sure that your assessment will work for your students. It is your responsibility to make any adjustments and confirm them with co-workers and supervisors.

You may have to adjust the assessment for:

1. physical disability involving hearing, vision, voice, mobility
2. intellectual disability
3. a medical condition such as arthritis, epilepsy, diabetes, asthma that is not obvious but may impact on assessment
4. differences in learning progress
5. reading or writing disability
6. denominational or other religious differences
7. cultural background or perceptions
8. age
9. gender.

As a reasonable adjustment, you might need to do one of the following:

1. take into account the student's language, literacy, numeracy requirements
2. give personal support (for example: reader, interpreter, scribe)
3. use special equipment
4. flexible assessment sessions to allow for fatigue or administering of medication
5. change the format of the assessment materials (for example, write them in Braille or the student's first language, use audiotape/videotape)
6. adjust the physical environment
7. revise your proposed assessment methods or tools
8. consider age and gender
9. consider cultural beliefs, traditional practices and religious observances
10. arrange for a member of the community to accompany the student.

Some assessment strategies are not particularly flexible. If the skill is to write a report, then the appropriate assessment strategy is a written report. However, the allowable adjustment might be that the report relates to a topic in which they have some expertise. Some competency standards specify what is not allowable as an adjustment.

Context

The most common kind of allowable adjustment comes from context. For example, an urban youth worker in a church might have different assessment needs to a government youth worker in a country town, though both could be assessed using the same youth worker standards and both might be equally competent.

Other examples are:

• fitting into family schedules
• assessment materials on videotape
• change of venue
• design of shorter assessment sessions to allow for fatigue or medication
• selecting assessment activities that fit their needs or backgrounds
• use of equipment such as a word processor or lifting gear
• allowing special techniques for lefthanders
• using alternative ways of gathering evidence (simulations, role plays, oral assessments)

Culture and language

You can make allowable adjustments for culture and language:

• allow work to be done in languages other than English if you have an interpreter or speak that language, and if English is not part of the requirement
• allow other teaching-learning styles
• identify different cultural expectations in learning.

Disability

Assessors need to find out whether they need to adapt the assessment to the student's disability. For example:

• A blind person taking a written examination might need a reader, or Braille documents and a Braille writer, a large print version of the paper, or audiotape. He may need either a scribe or a word processor.
• A deaf person may need an Auslan interpreter.
• A dyslexic student may take an interview rather than a written test.
• A sick person may need an attendant carer.
• Students may use a scribe or a word processor if their disability makes them unable to write.

Some kinds of adjustments suit disturbed or intellectually disabled students:

• Increased time (unless efficiency is a requirement)
• Using the same activity for both practice and assessment
• Integrating assessment into regualr class activities to minimise stressfullness.
• Allowing student to take practice assessments before the actual assessment to minimize stressfulness. These may be informal assessments, peer assessments or self-assessments.

Limits of reasonable adjustment for disability

Under the terms of the Act, a reasonable adjustment for a disability may not cause you "unjustifiable hardship". This is defined in terms of the benefit or detriment is likely for the student, the effect of the disability, and the financial cost to you. This means:

• You don’t have to give an adjustment if it won’t make any real difference. This applies to cases where an adjustment provides little benefit for the student, or where lack of an adjustment provides little detriment for the student.
• You must give an adjustment that provides significant benefit for the student, as long as it does not incur unfair expense to you. If the adjustment imposes unfair financial cost, then it is not required (For example, a small organization cannot be expected to hire an extra personal tutor, provide specialized extra equipment, or produce a special edition of texts.)
• If the disability is so serious that the student cannot perform the program requirements within allowable adjustments, you don’t need to make unrealistic allowances. You must give the result not yet competent. For example, if a blind person shows up for visual arts, you don't have to pass them.

Higher qualifications

When qualifications are driven by knowledge and concepts, skills usually require considerable reflection and take longer to develop.

Students' practical assessments for higher qualifications are more difficult than lower qualifications for these reasons:

1. Students usually work under minimal supervision and the immediate superior is often at a different location. Getting the supervisor to take a role in assessment is not very helpful.
2. The assessment timeframe is difficult. People at this level of responsibility are decision-makers. But it's a long time between making a major decision and getting results that can be used to evaluate the decision.
3. Assessment requires more expertise. Assessors must be competent subject matter experts, with both theoretical and practical knowledge.
4. The students’ work is conceptual. But knowing a static chunk of knowledge is not as helpful as being able to find and evaluate knowledge and to generate new knowledge.
5. Students work in various scenarios with changing variables. Just knowing how to do something isn’t enough; they need to know how to do it differently in different situations. If you ask them how to do something, they’ll begin with: It depends on … Sometimes they must plan for situations that can only be envisaged or forecasted.

Using third parties to gather evidence

With thanks to Russell Docking

In vocational and professional education, using local supervisors to gather evidence can cover hundreds or thousands or hours of work, more than an off-site college staff member could gather. They are not qualified assessors, so they are not actually making the assessment judgment, just gathering or providing evidence.

Orientation

To start, give orientation to supervisors to tell them what to expect.

1. Give supervisors a basic idea of Competency Based Assessment. CBA is fairly easy to understand: We need to know if they have the skills to do the job properly. If they can't actually do it, it doesn't count.) This is usually fairly easy except when they have ingrained preferences to higher education (They have to understand it. Doing the job is not so important.)
2. Give supervisors specific guidelines. They will need to know what you expect and when. Some will need deadlines. Any information needs to be in normal workplace English.
3. Do some moderation exercises. For example, you could show them examples of satisfactory and unsatisfactory performance.
4. Talk about records:
1. Provide record books that are ready to be filled in.
2. The records should be contemporaneous, done at the time of observation, not filled in from memory just before you show up.
3. The particular skills or tasks should be signed off, not the individual criteria.
4. Supervisors should write comments on what they saw, what the student did well, what student found difficult, etc.
5. They should be signed and dated.
6. Show them examples of record books filled in well and those filled in badly.
7. Anticipate that some supervisors will probably be surprised at the idea of good performance but poor records, and poor performance but good records. If the student's performance is poor, you'll need records that would hold up on appeal. There are four basic possibilities, and you might want to have examples of each:
• Good performance, good records
• Good performance, poor records
• Poor performance, good records
• Poor performance, poor records.

Ongoing Support

You will also need to give supervisors ongoing advice and support. Many suffer in silence if they get stuck, waste lots of time, and either over-record or under-record.

Plan to visit or telephone periodically. Ask how they're going and for any questions they might have. They might also unsure and lack confidence, and simply need to be old that they are doing it right. If you visit, see what they do.

As part of your moderation, you might want to have a co-assessment session during a visit on the job.

Other evidence

You might also need to discuss student performance with the supervisor, which you would record as an interview for extra evidence if the supervisor is averse to writing. The workplace will probably generate other kinds of naturally-occurring evidence such as job descriptions, log books, reports, products, and work evaluations. Obviously, you'll want to put it into a portfolio.

Information that students might need

You need to explain clearly to students a considerable amount of information on how assessment will be done. It must be provided before you start assessing, and often before you start teaching. It can be given in different ways:

• Some information is included in your college’s and general student information and policies.
• You can give some information to a whole class.
• You need to give some kinds of information to the individual student.

The kinds of information are as follows:

1. The purpose of the assessment, usually by mentioning the units or a qualification (i.e. the level and name) he/she would get upon satisfactory performance.
2. The date, time and place of assessments. If these are negotiable, then you need to be sure that both parties are in agreement.
3. Exactly what will be assessed.
4. How it will be assessed (i.e. the methods/modes, tools, and procedure, and the kind of evidence that will be gathered.)
5. The criteria against which they will be assessed (which may be implied in the teaching.)
6. Particular evidence requirements
7. Any relevant instructions and regulations.
8. Legislation and other regulatory requirements that affects the student.
9. Any standards for higher levels of competence, (e.g. graded assessment).
10. Explain any legal or ethical responsibilities to the students.
11. What happens if the student is considered competent or not yet competent.
12. How many chances they have to be assessed (usually another attempt).
13. If the assessment is unfair in some way, they can appeal.

Other information

When you start a new unit, you'll probably need to orally explain the outcomes to students. They often don't understand a written statement of outcomes because they haven’t studied them yet.

You might be well advised to mention several other matters:

1. Any formative assessments done as part of learning experience
2. Check that the student is correctly enrolled; it is seldom necessary but incorrect enrolments have prevented students from receiving qualifications.
3. Prevent misunderstandings about payment of fees because fees are often linked to enrolment.

With the possible exception of a campus group, you might need to include more information in the assessment plan. This would involve cases such as:

• several on-the-job assessments
• getting permission from a supervisor
• scheduling other people
• scheduling equipment
• documents that are difficult to get or should include particular wording
• special safety issues
• the list of assessment criteria is quite lengthy and complicated.

You rcollege should explain the following to students when they are admitted, but you might find them within your role as instructor or assessor:

• The accreditation status
• The assessor’s qualifications
• That the student has the right to view his/her file
• Workplace assessment policies
• Information on the college
• The appeals process

When students have extreme stress

Some students get extremely stressed about assessment. Imagine what happens from the student’s viewpoint. The college gives you an official-looking piece of paper describing the difficult things you have to do, the deadlines, the exam with the secret questions, and how you could ask for help after you fail. In fairness, the college meant well; it tried to inform students what's happening: how they'll be assessed, the times of the assessment, and how to appeal. But this kind of communication can simply build up students' dread of what might happen.

Students are considered educationally wounded when their bad memories of school negatively affect their ability to learn and to be assessed. They are averse to formal education and anything that looks like school, and generally believe that it sets them up to fail. Consequently, they see assessment as very daunting.

But many of these students are quite able to learn effectively. Some just don't like exams. Many fear failure and feel that the assessment system sets them up to fail. Some have unpleasant memories of school.

As an assessor, students' assessment stress affects what you do in various stages:

• How you plan your lessons to integrate with assessment
• How you inform students about assessment
• How you write assessment tools
• How you do the assessments.

Here are several tips to minimize student assessment stress.

When you inform students in writing how they will be assessed, make your written statement simple and easy to understand. Give it out early so it doesn't build fear. It might even help if they lose it. And talk them through it to allay their fears.

When you teach, help students them keep focussed on the goals. Verbally let them know what they have to be able to do and how well they have to do it. Most students are trying to get it right.

Integrate learning and assessment seamlessly. You create lots of stress when you mark the assessment time as The Big Exam.

Use the same kinds of activities for assessment as for teaching.

Just being "competent" isn't always enough. Students might not be ready for assessment even if they can actually perform the competencies. Give them time to build up some confidence. Wait until they are ready for assessment and you are fairly sure that they will pass comfortably.

Try stealth. In many cases, students will pick up a variety of skills through one learning activity. But they'd be terrified if we started by telling them that they'd have to learn all those things. They'd give up before we started. So we give them a learning activity that they think is quite achievable. Then after they've done it and we've assessed it, we say, Lo and behold! You've learned all these other things as well. You'll just have to get the extra credit for them. (With thanks to Alison Wright of ASRITC.)

You have more options in vocational and professional education:

The ways in which you assess required knowledge will vary according to the actual item to be assessed:

1. Most preferred: Depending on the knowledge to be assessed, students might be able to demonstrate it by doing the assessment task. This doesn’t even require a separate assessment activity for knowledge. Then, when students have done the assessment task, get them to explain what they have done and why they did it that way.
2. Use interviews. Start with the easy questions, and keep it practical. If a student doesn’t quite get it right, talk them around and give them another go.
3. Try to avoid written tests.
4. Required knowledge can mean either 'What someone needs to know to do their job in their organization' or 'An industry-wide body of knowledge that is found in standard textbooks.' When you write questions for assessing required knowledge, phrase your questions to address the first as much as possible. It is usually less stressful for students to describe the familiar things they see every day.

Make the assessment part of what they will do in their workplaces anyway:

1. Interview the workplace supervisor to report on work done in the workplace. You can sometimes even use the workplace supervisor as a co-assessor. The student doesn't need to attend the meeting.
2. Use workplace documents they've already done as assessment evidence.
3. Use a logbook to report what students have done.
4. Get the student to give you the walkaround tour of their workplace and explain what they do. They'll feel much safer on their own turf explaining something familiar. You might be surprised how much of the assessment can be done quite quickly.

At the end, you get the upside. First, everybody enjoys the assessment much more. Second, when you tell easily stressed students that they did well and have passed , they say Is that all? It was easy. And you can honestly tell them they earned it. You win too. You get an excellent graduate who couldn't have succeeded at your competitor colleges.

Your personal review

The main factor in reviewing what you have done is your personal fear. Nobody likes being reviewed or evaluated, and many people will do anything they can to avoid discussing their weaknesses. If you're new to teaching or training, or if it's the first time you've taught or assessed a particular unit, you'll be painfully aware of your mistakes. But you need to overcome those fears if you are to improve, no matter how good you are.

In practice, especially in a larger institution, a lot of what is done is to collate student feedback forms and your own forms, and to review the results with a responsible person or in a committee. Ideally, your assessment forms should have a space for comments that you can put into the review later on.

Say honestly how you think it went

At this stage, the main point is to say honestly how you think it went. Whether you taught a class or supervised work-based learning, you'll be painfully aware of what worked and what didn't, and you'll want to change some things next time. Don't forget to analyze your own skills. Don't just describe what you did but evaluate it. You need to answer What do you think needs changing, and how would you change it?

Hint: Make the review easier to do by making notes of good ideas and changes as you teach and assess. This will make the review more useful, because you'll include lots of things hat you would otherwise have forgotten.

You can base your evaluation comments on:

• the students comments to you,
• workplace supervisors comments to you
• industry representatives' comments
• your own reflections or observations
• comments from your colleagues, either incidentally or through moderation, or
• changes in the industry that you learned during the assessment

Other reflection questions

You need to be able to pick up on what went well and what didn't. The specific questions are not standardized but you might ask questions like these:

1. Was your delivery plan an effective guide?
2. Were unit expectations clear?
3. Was assessment appropriate to what was learnt?
4. Do students understand the subject better (not just know more)?
5. Was the workload reasonable?
6. Were facilities adequate?
7. Were written materials easy to follow?
8. Were instructions clear and adequate?
9. What new ideas did you have that you could use next time?
10. Did you get directed to other sources (books, websites, etc.)?
11. Were you accessible outside class?
12. Were you approachable and welcoming of your questions?
13. Were you informed of changes brought about by student feedback?
14. Were you well prepared for teaching sessions?
15. What worked best?
16. Could you teach some things more efficiently?
17. How could you be more effective?
18. How does the teaching plan compare with students' actual achievement of outcomes?
19. How do the program goals, the procedures and models compare with students' actual achievement of outcomes?
20. Does the lesson sequence make sense to students?
21. Do some things need to be added? (e.g. extra explanations or practice)
22. What would make the series more effective?
23. How effective have you have been in integrating work-based learning with external training?
24. What difficulties arose?
25. In light of the review, what are your areas for improvement and change?
26. What skills do you need to develop?

Make the changes

Identify areas you need to improve on and write them down. Then put the improvements into practice in your teaching and assessment. There's not much point doing a good review if you don't implement the findings.)

Formal evaluation

Your organization probably has an established review process. People in positions of responsibility and your peers might also need to take part in reviewing your work, and you should actively welcome input from students. The review processes their feedback on how well you did, and identifies areas for improvement.

Revew processess vary greatly between sectors. Almost all organizations have staff performance reviews, but those reviews might not cover teaching and assessment skills. Many higher and vocational education institutions evaluate teaching and assessment only through student feedback and complaints. In K-12 schools, trainee teachers are always closely supervised. Qualified teachers are only supervised if the Principal, Head of Department, or Superindent suspects problems. The trend in best practice, however, is to have a system where all teachers are encouraged to improve their skills and the quality of teaching and assessment in the school.

The simplest way to conduct a review is do this is to have a standardized form with clear instructions, and fill it in fully. Even so, you can never ask every good question.

Check that procedures and systems are working, and suggest changes if they aren't. Even if it's a good system, you might find ways to improve it. This means that you might need to report in writing to those responsible:

• the strong and weak aspects of the teaching and assessment
• substantiate your comments where necessary
• record and report any disputed decision
• ask whether the way you reviewed teaching and assessment was helpful and effective
• evaluate the records you keep to determine whether they meet the needs of the organization, and
• suggest improvements to those in responsibility

It is not acceptable to only report that you’ve thoroughly reviewed the unit or to say that the unit went very well and you wouldn't change anything. (However, good teachers can almost always think of changes they'd make next time.)

When reviewing the effectiveness of work-based learning, make a written record of work performance and learning achievements according to the requirements of your organization.

1. You need to ensure that your methods meet the principles of assessment. Compare your assessment process against established criteria or best practice guides. You may need your supervisor's advice on how to improve your assessments in the future.
2. Get feedback from your supervisor and your colleagues. They will probably be able to give you good advice.
3. Look at how you planned to do it and ask whether or not it worked. Did students meet the program goals?
4. Reflect on how you thought it went. Look at your notes and honestly evaluate your own performance.

Get feedback from your students

Feedback goes in two directions : from you to the students (discussed in another e-book) and from the students to you. For reviewing a program, we refer to the students' comments to you on how the teaching and assessment went.

You also need to encourage students to provide critical feedback on their learning experiences so you can evaluate the effectiveness of the learning program and the way it was done. Listen carefully to the student’s comments on how it went. It will make sure that you and the students understand each other and will allay any negative impressions that the students might have had. It will help in your review of the assessment.

Multiple choice student feedback forms are sometime sarcastically called happy forms because they often do little more that ask: Are you happy? They are widespread because they are quick and easy to use for both staff and students, and keep a paper trail of feedback. Although they meet any accreditor's requirements as a feedback system, they are often unreliable because people tend to give minimum answers with little thought. Reports vary; some responses are overly optimistic, and some are overly pessimistic.

Otherwise, you are quite free to look at a wide range of methods to evaluate what has been done. You don’t need to use all the methods, just those that are most helpful in your situation. For example, you might choose to interview management, instructors, assessors, and students.

Observing assessments could be most helpful. You might be able to see that a student is badly stuck on a task he/she finds difficult, but they might be unwilling to say so or mention it in feedback.

Other methods

Some other aspects of review methods are:

1. Record and address health and safety issues.
2. Outline how the sessions went. Base your answers on:
• students’ self-assessment
• student feedback
• comments from your supervisor or peers, and
• comparison against outcome statements.
3. Evaluate how you did getting students to:
• review how the teaching or training went, and
• review how the sessions helped them achieve learning goals. (Use the assessment results and personal observations to draw your conclusions.)
4. Evaluate how you did encouraging students to evaluate their own performance and identify areas for improvement.
5. Check student satisfaction though surveys of graduates
6. What were the students’ needs, and what actual progress did they make?
7. Get and discuss reactions from other relevant stakeholders. For example, you can check student performance though employer and industry surveys and feedback
8. Suggest ways of improving the procedure for recording information of the teaching and assessment processes.
9. You could also examine many different kinds of records, some of which might need careful analysis:
• exemplars and benchmarks
• guidelines to interpret endorse or accredited unit statements
• assessment tools, templates and proformas
• assessment records
• client and student feedback
• moderation results

Other possible questions

Here’s a list of the kind of things you might need to address in your notes with answers and supporting evidence:

1. Was assessment appropriate to what was learnt?
2. Did you handle the assessment gap successfully?
3. How long did it take to return assessed work?
4. Do students understand the subject better (not just know more)?
5. What do you think needs changing, and how would you change it?
6. How did stakeholders and students react? Is there documentation?
7. Did the instructors make changes to subsequent delivery based on feedback from students and others?
8. How many students were assessed? Over how long a period?
9. Under what organizational constraints did assessors have to operate?
10. What health and safety factors were significant?
11. How effectively did assessors relate to other appropriate personnel in the assessment process?
12. How often did you assess?
13. Did budgetary restraints affect the assessment process?
14. What information needs of government and other regulatory bodies were involved?
15. What were the support needs and professional development needs of assessors?
16. What were the characteristics of persons being assessed? Were they well motivated?
17. Are there any human resource management implications?
18. Were assessment decisions consistent?
19. How flexible was the assessment procedure?
20. Were assessments fair? Efficient? Effective? Reliable? Valid?
21. What competencies did students achieve?
22. What difficulties were encountered during the planning and conduct of the assessment?
23. Were the location and resources suitable?
24. Did assessments work well in their specified contexts?
25. What grievances or challenges to the assessment decision were there?
26. Is the system easy to administer? Is it practical?
27. Were there any access and equity considerations?

Title

Content 14

Title

Content 15

Title

Content 16

Title

Content 17

Title

Content 18

Title

Content 19

Title

Content 20