Competency-based assessment

Introduction

This guide to assessment was originally written specifically for vocational education, but most of it is just as useful for professional education. In vocational training and professional education, assessment is used to determine whether or not a student is competent in a particular set of skills.

Otherwise, the assessment procedures in this workbook apply to all kinds of learning.

The more general sections are suitable for K-12 schools and higher education but do not contain discussion of specific K-12 issues. Many higher education programs have different goals, teaching through lectures and tutorials, giving essay tasks as assignments, and using written examinations. If you teach and assess this way, you will need to see how these principles apply to you.

What is assessment?

Assessment can mean appraisal or judgment of worth.

In contrast, the term evaluation is used to refer to determining the quality of a whole program, so it now usually means something different from assessment.

One way to classify assessment is criterion-referenced and norm-referenced, and another way is as either formative or summative.

 

Many other kinds of assessment are also quite relevant to schoolteaching. Some of them reflect methods of assessment rather than purposes, and many are not mutually exclusive:

Diagnostic assessment
This kind of assessment is used to identify specific characteristics of students. For younger students, tests are used to assess learning readiness and stages of development. Tests can also be used to identify learning problems. Some kinds of assessments are highly specific and created by specialist psychologists. Others can be much more open-ended.

Reflective assessment
Students are asked to reflect on their learning experience. The results help students who need to "learn how to learn," and program designers can use it as feedback to evaluate program effectiveness and instigate improvements.

Scaling assessment
School systems often hold standardized tests based on generic criteria. The purpose is usually to check whether or not students achieve minimum standards, but assessment results are frequently used to create ranking comparisons between schools and between school systems. These assessments are normally unpopular with teachers because they reduce teaching to test preparation, focus on minimum standards, and ignore other kinds of valuable education outcomes.

Ipsative assessment
A student is assessed against his/her previous levels of achievement.

Synoptic assessment
Students combine elements of their learning from different parts of a program and to show their integrated knowledge and understanding of a subject.

Dynamic assessment
This kind of assessment measures what the student achieves when given teaching in an unfamiliar topic or field.

'Authentic' or work-integrated assessment
An assessment where the tasks and conditions are more closely aligned to those experienced in employment.

Embedded assessment
Students are assessed as part of regular teaching and curricular activities.

Cumulative assessment
Students are assessed at various stages within the program. It may be formative, ipsative, or summative.

Capstone assessment
Students are assessed at the end of the program though a major culminating summative activity that represents the outomes of the whole program. It may be a practicum, a project, or a thesis.

https://www.exeter.ac.uk/staff/development/academic/resources/assessment/principles/types/ June 25, 2018

 

Why do people take assessments?

People take assessments for many purposes, for example:

• to determine what they have learnt during a course
• to determine if they are making good progress in a course or job
• to find out their skills in language, reading and writing, or basic mathematics
• to get recognition of skills they have learnt on the job or informally
• to find out if they have gaps in their learning
• to get admission to a course
• to get a particular qualification or statement of attainment
• to get a license of some kind
• to measure their performance at work
• to become a member of a professional association
• to get a particular job
• to get a promotion
• to identify training needs and career paths
• to determine their employee classification

The most interesting point is that you could use the same process for more than one purpose:

 

Competency-based assessment

A competency is a course requirement that is expressed as something the student must be able to do and defines the actual requirements.

In school systems, they are usually written in the curriculum framework or a syllabus document. In professional or vocational education, competencies are witten in a competency standards frameworks. In other higher education, they are set by the particualr institution. They are often written in full in the unit description for each unit.

It is helpful to think of competencies more broadly. They aren't limited to performance skills; they may be conceptual, such as analysis, formulation, and strategy. Their written length does not necessarily reflect their difficulty. Some challenging tasks can be described quite briefly.

In professional and vocational education, the competencies of a degree together usually reflect a specific work role. The competencies of a purely academic degree define the level of mastery of an academic discipline. Options may allow for one degree to accommodate multiple roles.

No matter how well-written competency statements are, interpreting them always brings up questions. Competencies are umbrella statements and can't specify every micro-skill. Then again, they don't need to do so. Similarly, they can't be comprehensive, because writers can't predict every kind of serious mistake that students will make.

Interpreting a well-written competency statements doesn't need to be difficult. First, competencies are to be interpreted in the context of the generic requirements of the qualification and its purpose. Second, competency statements normally also include smaller objectives or criteria to help ensure consistent interpretation and to specify more clearly how well the competency must be done. For example, a competency may be written as a purpose statement and a list of skills, each with a set of performance criteria. A performance criterion can be any kind of indicator that shows whether or not the student has performed satisfactorily.

 

Implications

Some of the implications are:

1. Students must satisfactorily demonstrate all requirements of the comptetency to pass a unit. Achievement above the minimum for one part of the unit does not compensate for under-achievement in another part of the unit. In other words, one cannot "average out the marks" of different assessment activities. For example:
• Good performance in one part of the practical cannot compensate for inadequacies in another part of the practical,
• Good performance in a theory test cannot compensate for inadequacies in practical performance,
• Good performance in a practical performance cannot compensate for inadequacies in a test of required knowledge.
2. Competencies may be assessed in various ways as long as assessments are consistent with the competencies.
3. Assessment is fairer and often more objective.
1. It can work well for small numbers of students. (Normative assessment requires populations of students that is large enough to grade on the curve.
2. All students can pass if they meet the competency requirements. (In norm-referenced approaches, the bottom group of students must be failed at an arbitrary percentage.)
4. Insitutions can experiment with MOOC-like courses (low-cost highly automated tuition), as long as tutors can monitor students' progress toward specified course outcomes.
5. Institutions can offer:
1. tuition-based courses, the purpose of which is to achieve the competencies.
2. different kinds of tuition for different student populations, as long as they consistently address the same competency standard.
6. Institutions can offer the alternative pathway of direct assessment of competency. (RPL)
7. Requirements for higher grades can be defined more precisely and disclosed to students.
8. Students who achieve a high enough standard might also have evidence of competence for more advanced units in the same field.
9. The same item of evidence can be used to address multiple competencies. For example, a complex project could address competencies in the management of personnel, budgeting, finance, and projects.

In some jurisdictions, higher education programs must be defined in semester hours. In the US, a semester hour usually represents 45 hours of study. Consequently, programs can be defined in both competencies and semester hours. One semester hour represents the learning competencies achieved through 45 hours of study, based on the estimated average time for native English speakers in the program’s target group to achieve those competencies.

 

Recognition of Prior Learning (RPL)

What about people who have never studied? Can they just apply for the assessment?

Yes. Many vocational colleges and institutions of higher education allow for the recognition of current competencies regardless of how they are acquired. Terms vary greatly. "Challenge test" is quite common in the US, while Australian vocational colleges call it "Recognition of Prior Learning" (RPL).

People can learn skills though:

• a hobby
• a friend
• their own reading
• experience on the job
• involvement in church or other community organisations
• foreign study
• unaccredited colleges
• studies that were done before the college was accredited

If someone has learnt the skill through any of these ways, or any other informal way, and meets the requirements to get into the program, then they must be allowed to be assessed. They can also be assessed for RPL to gain admission into the program.

Training organizations may not refuse assessment because an applicant didn't sit the classes or training sessions. If the student passes the assessment, he/she is not differentiated in any way from those who sat through all classes.

RPL affects assessors because assessors now have greater responsibility. The assessment is the only way to ascertain whether the student has those skills. In the past, the classroom teacher could monitor student's development through assignments, class activities and formative assessment, so they had a guideline to know whether assessment went well.

Without classroom monitoring, the assessor is responsible to ensure the quality of assessment, and assessment becomes mainly summative.

RPL has another effect. RPL students and regular classroom students must sit equivalent assessments. It is unethical to make one more difficult than the other, for example, an assessor can't say that because an RPL student's skills weren't monitored in class, their assessment may be much more difficult.

In the past, some institutions used unethical tactics to make sure that students could not pass an RPL assessment. They did this by:

• making fees for RPL as much as for tuition and assessment
• making the RPL more onerous than taking the class
• failing to offer RPL to applicants
• failing to have people knowledgeable of RPL procedures on hand
• writing competencies and assessment tools to include lengthy formal essays or in-class group work
• requiring students to agree with a particular philosophy of the subject, and/or
• requiring students to perform the skill in a particular way and disallowing other equivalent ways.

RPL is not the same as transfer credit. The confusion can arise because one way to do an assessment is by a portfolio of documentary evidence (discussed later). Besides, it becomes possible to recognize unaccredited study through RPL, as long as there is sufficient evidence of learning and an assessment is made. RPL is quite different from transfer credit:

• RPL requires an assessment. It normally involves substantial work, so students must pay a fee.
• In transfer credit, the college recognises certifications issued by other institutions, although recognition between institutions varies greatly.
• Some institituions have transfer credit agreements to ensure that transfer credit meets all requirements of the recieving institution; students only need to show the certification and no assessment is necessary.
• In most other situations, nobody awards full transfer credit for everything. Proposed transfer credit is assessed against its accreditation status and the requirements of the program to which it might be transferred. (The latter usually results in some non-recogntion, because higher education programs are usually designed to be unique in some way.)

Portfolios of documents are still quite useful in some circumstances, but the trend is generally away from them. Current RPL procedures are more hands-on and on-site, including focused interviews, observations and walk-throughs.

RPL: Like a job application

Here's an interesting thought on assessment. A Recognition of Prior Learning assessment can be much like applying for a job:

Applying for a job

Assessment

• Your CV outlines your experience and achievements with enough detail to convince the prospective employer that you are competent for the position. You list your qualifications and any other training, and you enclose references and a portfolio of work samples. • You address each selection criteria, showing how you meet it. • The prospective employer verifies what he/she reads by checking the references with past employers • In the interview, he/she sights the originals or certified true copies of qualifications. He/she also asks questions to test whether you really have the skills for the position.

• Your CV outlines your experience and achievements with enough detail to convince the assessor that you have the competencies of the qualification. You list your qualifications and any other training, and you enclose references and a portfolio of work samples. • You address each element, showing how you meet it. • The assessor verifies what he/she reads by checking the references with past employers. • In the oral assessment, he/she sights the originals or certified true copies of qualifications. He/she also asks questions to test whether you really have the skills for the qualification.

Tip: People who learn on the job probably structure their knowledge in very different ways from textbook writers and class-room students. Some think they don’t know the theory but can usually give you very good reasons for their decisions and why they do it the way they do. Sometimes you need to re-phrase the question to suit the way they think.

Step-by-step procedure for
Recognition of Prior Learning (RPL) students

Portfolios of documents for RPL are still quite useful in some circumstances, but the trend is generally away from them. Current RPL procedures are more hands-on and on-site, including focused interviews, observations and walk-throughs.

Here are two different step-by-step procedures for RPL, one by portfolio and one by practicum. I've put them side by side so they are easy to compare:

 

Portfolio

Practicum

Prospective student approaches the RTO. RTO informs the prospective student about RPL for admission, units, qualifications, etc. RTO assists the prospective student to write a full Curriculum Vitae and/or do a self-assessment. RTO gives advice on what the prospective student should apply for (qualification, statement of attainment, admission). The prospective student for qualifications might need to take some units through instruction (non-RPL). The prospective student applies to become a student. Assessor plans the assessment (Identifies the evidence that will be needed, appropriate assessment tools and strategies, trials the assessment procedure if necessary) Student prepares a portfolio of evidence with help from the assessor Assessor assesses the evidence and documents the process. Assessor gives results and feedback to the student. RTO provides the qualification or statement of attainment or confirms admission to desired program The RTO reviews and monitors the process.

Prospective student approaches the RTO. RTO informs the prospective student about RPL for admission, units, qualifications, etc. RTO does a preliminary estimation of the student's abilities. RTO gives advice on what the prospective student should apply for (qualification, statement of attainment, admission). The prospective student for qualifications may need to take some units through instruction (non-RPL). The prospective student applies to become a student. Assessor plans the assessment (Identifies the evidence that will be needed, appropriate assessment tools and strategies, trials the assessment procedure if necessary) RTO places the prospective student in a practicum placement. Depending on the kinds of skills to be demonstrated, it might range from several hours to several weeks. Assessor assesses the student in the workplace. Assessor gives results and feedback to the student. RTO provides the qualification or statement of attainment or confirms admission to desired program The RTO reviews and monitors the process.

 

The assessment gap

Education policies perceive a gap between:

Curriculum statements of outcomes and assessment criteria	and	Making specific assessment decisions with real students

They are concerned about assessment decisions being judgment calls made by assessors. For example, if a student appeals, can you prove your assessment was correct? Or was it just your best guess?

A great deal has been done to close the gap and take any guesswork out of the assessment decision. Nevertheless, solutions to the guesswork problem come up as a constant theme throughout assessment practices:

• validation or moderation of assessment
• requirements that are increasingly specific and explicit (performance criteria, critical aspects of assessment)
• written program development reports
• written assessment plans
• itemised forms
• getting students to sign assessment plans and results sheets
• getting industry approval of specific assessment plans
• etc.

 

Holistic assessment

Some authorities tend to prefer holistic assessment.

Could you have one assessment activity that gets evidence for many related elements? Holistic assessment means that one assessment of one complex activity might cover more than one element or even more than one unit. Over-assessing is costly for you and frustrating for the student.

You might be able to save yourself and your students a lot of work by clustering assessments. The point is that students must be able to put all the different skills together to be able to do the job:

Example 1.: A receptionist might be doing one job at the desk and be assessed for separate skills in workplace safety, communication, information management, and computing.

Example 2.: A factory worker could use a piece of equipment correctly while complying with workplace safety rules.

Holistic evidence is good. A good holistic assessment is based on actual practice in a real workplace or a realistic simulation. Being part of one work role, the elements have common circumstances and probably overlap; students often perform skills from different units simultaneously.

In fact, a single assessment activity can combine knowledge, understanding, problem-solving skills, technical skills, workplace safety, language, literacy, numeracy, people skills, attitudes and ethics.

But you can't make a "holistic judgment." It wouldn't be responsible to make a gut feeling assessment judgment (Um, yeah, he/she can do the job) based on a whole-of-life report consisting of qualifications, training, and experience.

The problem is that the specific written requirements and the person’s performance don't clearly correlate. In other words, the assessment gap is too wide and the assessment is basically guesswork. To solve this problem, you should assess according to specific requirements of the units.

 

Possible assessment results

The result for each vocational unit will be that the students are either Competent or Not yet competent. Many vocational units are well suited to these assessment outcomes, and most assessors appreciate the simplicity.

Grades always need defining. In a letter grading system, A usually means outstanding, B means excellent, C is a pass, and D and E mean that the student has not passed.

Grading is normally built in to norm based assessment, and works well when students numbers are large and the assessments are done very consistently. Note the conditions; it doesn't work for very small numbers of students or if the assessment system cannot ensure very high levels of consistency.

Many institutions give grades as A, B, C, D, or E. Students, schools, employers, and universities all want it. Grades are necessary for students transferring to the further studies or internationally. Consider this scenario:

 

Percentages in assessment

Based on "Percentages in assessment" by Ross Woods

"What percentage constitutes a passing grade?" For example, some people might say 60%, others might say 70% is higher, and others suggest that 80% represents "higher quality" education.

The fact is that the actual percentage figure is completely arbitrary. An assessor could easily dumb down all questions in order to require 100% correct. Similarly, an assessor could make them all so difficult that 20% correct is a good result.

In norm-based assessment, what really matters is the percentile needed to be deemed a particular grade.

In a competency-based system, however, it is the other way around. First, write competencies to define a passing level of performance. Then set the testing system so that a pass fits the competency. In competency-based assessment that has lists of required knowledge, all knowledge is required so we can legitimately ask: "If it doesn't matter whether students get a particular question wrong, then nobody needs to get it right to pass. So why are you asking it?" In other words, it is quite legitimate to ask for 100% in these cases.

Some systems have it both ways; they use a rubric for assessors to classify answers as "unsatisfactory," "satisfactory," and one or more levels of exceptional. Other have several stages:
• Early questions only settle students into the testing environment.
• The next batch of questions tests the basics, and represent the competency standard. Students must get them all correct to pass.
• The following batch of questions are more difficult; their purpose is to enable students to get a grade above the minimum pass.

In higher education, the system of graded assessment is often left to each particular institution. Oddly enough, It is most common in research work, where theses aand dissertations are usually assessed on some kind of pass / not pass system. "Not pass" often means that corrections are required, a lower qualification is awarded (such as a Master of Philosphy instead of a Ph.D.) or the student leaves with no qualification.

In other cases, the student must demonstrate satisfactory competence in all requirements, and a grade can then be given on the extent that their performance exceeds the standard.

The way to derive a letter grade is to define requirements for letter grades. These can be done on a micro level for each assessment, perhaps as a rubric. They can also be set at institutional level, such as the following:

Grade	Meaning
A	Outstanding achievement, competent at the next highest qualification
B	Meets the minimum requirement for recommendation to proceed to the next highest qualification
C	Satisfactory work to pass in the qualification for which the student is enrolled, but not recommended to proceed to the next highest qualification
C-	The minimum requirement to pass
D	Does not meet the minimum requirement to pass
E	Very poor

 

About evidence

Assessment also implies that there is some evidence that can be assessed. Evidence is information, materials, or products that show whether or not a student has the skills. For example, a teacher could watch them directly and record an assessment result. In a vocational training or professional education context, an assessor could get reporting forms of what the student has done, or get references from work or practicum supervisors.

The evidence is usually a direct correlate of the assessment mode. There are three kinds of evidence, although assessment authorities seldom agree on definitions:

Kind	Key feature	What it means
Direct evidence	Direct observation in a real situation.	As assessor, you directly observe the student performing the skill in a real or simulated situation. (To be called a simulation, an assessment must be realistic enough to qualify as direct evidence.)
Indirect evidence	Assessor infers competence.	As assessor, you infer competence from what the student has done (such as samples or written work). This may also include classroom-based assessments, and performance of conceptual skills in an interview or a test.
Supplementary evidence	You rely on someone else to inform your decision.	This includes third party reports, references, and professional licenses. However, the assessor is still responsible for the final assessment decision.

Supplementary evidence is excellent and in no way inferior. Original signed documents with detailed statements of competence from responsible, competent persons are very appropriate. Those documents need to be either issued independently or authenticated, and need to be free of conflict of interest. They usually take the form of references or professional licenses from credible bodies with established standards. It is good practice to follow up references with a phone call, because people will often tell you things in person that they wouldn't put in writing.

Of course, some third party evidence lacks detail or credibility, and is best used simply to corroborate other evidence.

Some authorities list the student's claims to competence (CV, self-assessments, etc.) in supplementary evidence. They are not real evidence and should not much determine assessment results. They are most useful for establishing an appropriate assessment. In a taught program, they monitor the students' readiness for assessment. In Recognition of Prior Learning, they indicate areas and levels of ability and possible sources of third-party evidence.

Most competency standards leave it up to teh asessor to determine which you will get. Some, however, require you to gather a combination of direct, indirect and supplementary evidence, or specify that direct evidence must be collected.

 

A code of practice for assessors

Below are the main points of a code of practice for assessors, most of which reflect ethics in some way.

1. Identify and handle needs that differ between students and between contexts.
2. Avoid conflict of interest; refer students if necessary.
3. Avoid harassment.
4. Protect student's rights.
5. Do not let your personal and interpersonal factors that aren't relevant to the assessment influence the assessment.
6. Inform students of their rights and how to appeal.
7. Base your assessment decisions on evidence that could be verified by another assessor.
8. Comply with policy and procedure guidelines.
9. Agree formally with students that the assessments are carried out according to agreed procedures.
10. Comply with equal opportunity legislation.
11. Tell students of all reporting processes and all known consequences of the assessment before the assessment.
12. Maintain confidentiality.*
13. Release results only with the written permission of the student.*
14. Use assessment results according to the purposes explained to the student.
15. Assess yourself regularly as an assessor.
16. Take professional development opportunities.
17. Network with other assessors.
18. Get technical help when you need it.

*Careful, confidential use of assessment results is imperative; an unwisely released "Not yet competent" result can have severe career ramifications for the student. There is no special exemption for the student's parents or employers, even if they have paid for the course.

 

Ethical issues

Conflict of interest

1. Does the assessor have a personal relationship with the student other than a student-teacher relationship? Assessing a family member is often considered unethical. But where do you draw the line if you live together on a residential campus? You might be called upon to assess close friends.
2. Will the assessor get a reward in terms of money, power, or job opportunities?
3. Assessors in any institution that depends on getting new students student fees and public goodwill feel some pressure to give a favorable assessment result. Institutions don't survive long if assessors fail most of their students, even if passing incompetent students is to nobody's long term advantage.
4. Has the assessor already received something that will make them feel indebted or that they owe a favor?
5. Will the assessor lose something if they don't give a favorable result?

Identifying your limitations

Your surgeon might be licensed to give you brain surgery, but would you want him to do it if he knew that he couldn't?

Don’t assess outside your area of ability even if you have appropriate qualifications. If you know that a topic is outside your current abilities, then it is normally unethical for you to assess it alone. You might get advice or have a co-assessor.

Find out your own limits. For example, you may have limitations related to your ability in assessment procedures, quality processes or your own competency level. There can also be related legal responsibilities that you cannot meet.

Your college might also put some constraints on you:

• assessment policy and procedures
• your job description
• quality assurance and continuous improvement procedures
• business and performance plans
• collaborative/partnership arrangements
• your resource limits (budget, schedule, etc.)

Other ethical issues

1. You might have been so involved in helping a student assemble a portfolio or in supervising a project that you can no longer be the sole assessor. You can either get someone else to assess, or get their input to check your assessment.
2. In vocational and professional education, if you know of relevant industry standards, then you cannot honestly ignore them.
3. Students might carry out a procedure that uses human or live animal subjects.
4. There could be privacy or public liability issues.
5. Could accidents be possible?
6. How could you protect the intellectual property rights of submitted written work? Students might not own copyright on their writings if they wrote them as part of their employment duties.
7. Could submission of written work put its authors or their organizations at commercial disadvantage or disclose private information?
8. It is unethical to deliberately plan larger assessment activities than required for the units for which students are enrolled, or
9. It is unethical to require students to take extra units, or
10. It is unethical to charge students an extra fee for units that are necessarily covered in the assessment of units for which they are enrolled.

 

 

 

Attitudes

There is generally no assessment policy written on the role of attitudes, and some authorities take the narrow view of assessing behavior only and consider attitudes irrelevant. However, many competency requirements are de facto attitude statements.

The rule of thumb is that attitudes must be appropriate to the kind of skill. The role of attitudes varies with the kind of job:

• Some skills are not necessarily very people-oriented, but they still require punctuality, politeness, basic teamwork, a sense of personal responsibility, and a commitment to do a good job.
• Some tasks require the student to put aside any personal issues for the moment and act with confidence and poise to do the job at hand.
• At the other end of the scale, some skills require fairly deep commitment to a particular set of goals and ethics. Or they might require some fairly deep personal feelings appropriate to their skills (for example, counseling and other caring professions).

Besides, some attitudes (such as prejudice based on gender or race) can be unconscious but observable in behavior. Even when attitudes are unconscious, they still need to be appropriate to the student's work. Training should have raised those issues to a conscious level.

Confidence usually comes up as an issue.

Ideally, you'd like students to do a good job and be confident that they can do well. Consider these five variations:

1. Confidence is essential in some jobs, such as those that require students to make decisions quickly and handle difficult people, and then keep going afterwards without taking a stress break.
2. Some overconfident students don't do a very good job. You have to assess them as not yet competent and they usually resent it.
3. Some students lack confidence but do very well. You can assess them as competent.
4. Some students perform poorly because they lack confidence. It's a good idea to give them more experience and assess them again later when they are ready.
5. Some students lack confidence and are clearly not yet competent. You have to assess them as not yet competent. You might be able to give them more experience and assess them again later if they are able. You might also ask why they are in the course and consider re-directing them.

Of course, if you're assessing someone who lacks confidence, it's an excellent idea to encourage them as much as you can.

 

General standards

General standards are standards that apply to all assessments. Assessments must address several principles of assessment: validity, reliability, flexibility, and fairness. Evidence must be sufficient, authentic, and current. Some are less complex and we can deal with them briefly before progressing to those that are more complex.

Reliable

The basic meaning of reliable is that the assessment works the same way every time. Reliability depends on assessors sharing a common interpretation of the units and of evidence being assessed.

It means consistent results:

• from the same assessment tool
• in different contexts
• with different evidence
• by different assessors
• with different students.

Will the assessment give the same results at different times or for people who have learnt the skills in quite different ways? Will the evidence be interpreted in the same away by other assessors? Do different kinds of assessment of the same skill produce the same results?

Flexible

Will the procedure assess people in widely different situations?

Flexible means that the assessment works equally well for all students and situations for which it was designed.

Fair

Fair means that the assessment works equally well for all students for whom it is designed, including disabled students where relevant. Perhaps no single assessment strategy is equally fair for all students, although an assessment strategy can be fair for all members of a particular group.

Does the process favor one kind of student over another? How would you assess a disabled person? Do students know they can appeal? Do students know beforehand the way they will be assessed and criteria used in assessment?

Authentic

Is the work being examined the student's own work? Do you need to verify it? For example:

• A written report where the majority of the discussion was ‘pasted’ from the Internet is not the authentic work of a student.
• If they student was required to write the report themselves but most of the thinking was done by their friend, then the report is not really their own work.

Current

Are you assessing skills that the student has now? Or what they once knew and have since perhaps forgotten? For example: A 10-year-old report confirming a student's computer literacy is not evidence of their current skills.

Some people also now use "current" to mean that you are assessing against the current version of the standards. While it is not too difficult to keep track of competency standards, some national bodies frequently make minor revisions that are difficult to track.

Valid

Valid means that the assessment assesses what it is supposed to assess. This has two meanings:

Educators tend to define it as assessing the right thing. Two of the most ommon examples are as follows:

Another meaning of valid, often used in vocational education, means that the assessment complies with the competency standard and addresses all its requirements. In this meaning, assessments are invalid if they don't address all competency requirements. Vocational educators vary in opinions as to whether adding extra requirements makes the assessment invalid.

Does the assessment adequately cover the range of skills? Does it integrate theory and practice? Are there multiple ways to assess the learning?

How much evidence is sufficient?

It's the assessor's responsibility to have a procedure that asks students for enough evidence. It is then the student's responsibility to provide enough evidence.

There is no universal definition of 'sufficient evidence.' Some competency standards make your job easier; they actually define it (e.g. "on three separate occasions"), so you only have to comply with that definition. (There is still a potential trap. Unfortunately, some competency standards routinely specify in most units, "If a number is not specified then one is adequate," although a single shot might not really be enough.)

The best practice for most cases is as follows:

1. There should be at least two independent kinds of evidence for each competency requirement. Each assessment mode has its limitations, so it’s not ideal for both items of evidence to use the same mode of assessment. A single mode can seldom cover all kinds of required learning equally well. (Even directly-observed abilities usually need corroboration through oral questioning and workplace supervisors.)
   Besides, a single assessment activity seldom suits equally all learning styles. Some students do better at oral assessments, and some do better at practical or written work. Oral presentations favor confident speakers but disadvantage students who fear public speaking.
2. Evidence should cover an extended period of time. This reduces stress on the student from risking all in a one-shot test. In vocational education, a practicum is preferable.
3. All evidence should consistently indicate competence.
4. Evidence items should corroborate each other. The two items of evidence usually come from different perspectives. For example, you might observe the student for an assessment. The student’s supervisor might then provide a reference based on the outcomes and mentioning consistency of performance over a period of time.

But ...

There is no universal definition of ‘sufficient evidence.’ It can vary according to the actual competency, so you will need to find out how much evidence is sufficient on a case by case basis. Consider the following ...

1. Some skills need assessing over a period of time and in many contexts. For example, students might have to handle many different kinds of people with widely different problems, including at times when they might be tired or stressed. That is, a report of continuous performance monitoring over a period of time could reflect performance on a number of occasions and in a variety of contexts, yet be one kind of evidence and one assessment.
2. Some skills are very straight-forward. They might be performed only a couple of times if the requirements are quite consistent and students can't fluke it. Assessment doesn't require an extended period of time. The competency statement might also specifically allow only one context.
3. Lots of weak evidence (e.g. poorly done observations) may still be insufficient.
4. A major project needs to be done only once. A Ph.D. student only has to write one dissertation; you couldn't require a second one. The extended time period covers various contexts.
5. High-risk skills need more evidence. In vocational education, this is more common in industries where students face many kinds of risks (financial, legal, health and safety, etc.) Would you want an unsupervised brain surgeon operating on you if he'd only ever done it twice?
6. Some competencies imply certain kinds of evidence. For example, the competency "Write a report" requires a written report as evidence. (Every now and then a competency might allow only one kind of evidence.)
7. Don't count self-ascriptive evidence. This refers to things the students claim about their skills without actually demonstrating them, for example, an unverified CV. A claim to competence is not the same as competence. However, it is quite permissible to have it as extra evidence.

Assessment of knowledge is a little more complex:

• Some requirements are only assessed as knowledge. For example, first aid students only have to know what to do in some kinds of accidents, but don't have to demonstrate those skills.
• Assessing theory doesn’t count for applied skills. Don’t use an assessment of theory as a second source of evidence because it is not an assessment of competence, even though you also have to assess required knowledge.
• Assessing theory does count for conceptual skills, such as solving complex problems. The difference between knowledge and competence is less distinct in studies that are more conceptually driven, for example, many school and university courses.

 

Vocational and professional generic requirements

In vocational and professional education, several another de facto requirements are not written in policy but derive from the idea of competence. In essence, they mean that the student has all the skills they need to do their job relative to the competency requirements. Your students need to learn more than just the skill that is the main point of the lesson, and you are usually required to reinforce them in your teaching and assess them.

Various contexts

Vocational and professional students should demonstrate competence in various contexts. The most common way to provide various contexts is to assess over an exended period of time. Other ways are to move students through different roles or locations, or to assess their skills using different equipment, etc.

Can they use what they've learnt in other contexts? Can they apply it outside the narrow confines of what they are doing at the time? How broad are skills? On one hand, students need to be able to transfer skills to other situations. For example, a hairdresser who can only cut blond hair is inadequately trained, and he/she should be able to use those skills on non-blondes. On the other hand, students don’t necessarily need to perform the skill in every possible situation. The hairdresser need not perform every possible haircut to demonstrate competence.

When doing an assessment, ask, "What would you do in a situation where …?" "How would this apply to …?" "Some organizations have a rule that …. What would you do if you had to perform this skill there?"

Here are a few simple solutions:

• The assessor observes the student do the task, and asks questions as they go. The assessor then gets a reference from the workplace supervisor for a period for several weeks or months.
• The student does a major project, submits the workplace paperwork, and does an interview.
• The student does two or three substantial tasks, submits the workplace paperwork, and does an interview.

Consistency

You should require consistency. The trend is that all evidence should consistently indicate competence, so it is clear that the student can perform the skill consistently on more than one occasion. A patchy performance should not be considered competent.

The rationale is that a competent person has consistent performance. In fact, some standards now explicitly require consistency. Depending on how much evidence is sufficient, you may have to check that the student has performed the skill in a range of contexts over a period of time.

If you face inconsistent evidence, check to see what is going on. Here are two examples:

Workplace requirements

For vocation and professional education, students must demonstrate competence at a workplace standard, even though standards vary from workplace to workplace. It assumes that your graduates must be employable, and employability can be used as a standard.

This has two implications:

1. If the student can perform the final skill at an amateur level but could not get or keep a job with those skill levels, it probably means that the student is not yet competent. This might particularly affect volunteers in training. In principle, they should normally be proficient enough to be employable.
2. If the student can perform the skill in an artificial campus situation but not in a real workplace environment, then the student is probably not yet competent. That is why simulated contexts must be quite realistic.

Legislation and regulatory requirements

Students should know the relevant laws with which they should comply. And if they do something illegal during an assessment, they must be assessed as not yet competent.

Professional and vocational colleges must maintain compliance with legislation and any regulatory requirements. That is, compliance is required by the college whether or not the competency standards says so. This compliance requirement extends to generic things like WHS, anti-discrimination, and consumer law. If the standard doesn't explicitly say so, compliance with legislation and any regulatory requirements should be read into it.

Workplace health and safety is a regulatory requirement, so you are also entitled to require that students comply with WHS rules and you can ask about relevant WHS aspects of the task in an assessment. In fact, you are legally required to maintain a safe environment. As part of an assessment, you may be able to get third party reports from the student's supervisor that the student complies with WHS procedures, or include WHS in your observations.

Task skills

These are simply the ability to handle individual tasks. Can the student actually do the job? You can use this requirement to justify the little extra items that are part of doing the job.

It can be necessary to include efficiency here. That is, a student isn't competent if he/she can't perform the skill within a reasonable amount of time. For example, a new trainee nurse might do just as good a job washing a patient as a competent nurse, but take twice the time.

Manage contingencies

Do they know what to do if something goes wrong or not to plan? How do they handle interruptions?

Ask lots of "What if …?" questions. "You depend on person X to get their job done correctly. What if they made a mistake?" "What if this machine broke down? What would you do?"

"What potential problems normally occur in this situation? How would you respond to each one? What are the signs of it becoming a real problem, not just a potential problem? When would you try other ways of doing this?

Handle their job/role environment

Do they know how to relate to other people? How do they relate to their organization?  If they have to do paperwork, do they keep it up to date?

Manage themselves and their tasks

Gather evidence on whether students can manage themselves and their tasks. For example, can they plan their own work, predict consequences, and identify improvements? Ask about getting people or equipment organized. Ask how their schedule works, and whether they keep paperwork under control.

Title

Content 15

 

Title

Content 16

 

Title

Content 17

 

Title

Content 18

 

Title

Content 19

 

Title

Content 20