Assessing units over the Internet
Ross Woods, 2005 (ed. Feb 2006, Jul 10)
The main problems
So far, the main problems with assessing over the Internet are:
- It is difficult to be sure that the work submitted is the student's own work.
- Tests of knowledge or aptitude often depend on the student not having access to questions beforehand, working in "exam conditions" or working without materials from which they could cheat.
- It is easy to assess knowledge but nearly impossible to assess competence.
This hypothetical system runs through on an online database, and depends on having at least one trustworthy person on the spot. They might be an assessor, an evidence gatherer, or a workplace supervisor who can verify the work submitted.
The system of permissions means that only the assessor can verify the results, and enter them in as an assessment result. Moodle's system of assessment records already has this capacity.
In fact, some assessors are already using a very similar system using PDAs that they upload to a server.
From another perspective, we could say that it's not really any different it's just that assessors and evidence gatherers feed assessment information into the database over the Internet.
It depends of having a secure server with a secure system of permissions. Staff use their passwords as an electronic signature and don't pass them on to anybody.
A hypothetical system
Here is a hypothetical system that could run over an online database. In fact, some assessors are already using a very similar system using PDAs that they upload to a server.
The systems depend on having at least one on-the-spot assessor. Only the assessor can verify the results and enter them in as an assessment result.
The staff member's electronic signature comprises:
- A system of permissions to ensure only staff can write assessment records.
- Secure access to a secure server through a password
- The user ID. and time are written automatically by the server
When assessment tools are revised and uploaded once for on-line use, the revised versions are immediately available to everybody authorized to read them.
The way the database is set up, with varying numbers of data fields with different kinds of alphanumeric entries is of course quite another matter. Any one system needs to be uniform for all kinds of students and all kinds of units.
Auditability is an interesting point. The secure logon is the main way to ensure that records are genuine. This may be inadequate in cases where the student is required to approve the assessment process beforehand, although that could be done in much the same way, that is, by approval given through a secure link.
Limitations
This doesn't really eliminate personal contact. In fact, it could be seen to be a strength of the approach that it depends on having someone on site.
There are limits to what people can learn from reading instructions on a computer screen, which appears to best suit text-based learning styles.
There are limits to how effective assessors can be without some kind of personal meeting with other assessors, and assessors still need training in assessment procedures. If it is easy to "tick and flick" with a paper form, it could be easier with an electronic form. No matter how good the system is, it still needs responsible people.
It can be quite complex to have different assessment tools for the same unit, which happens if you need to offer it in different contexts. However, it need present no particular obstacle, as long as they are equivalent.
How much is it a full-blown assessment tool and how much is it simply a recording system? I'd suggest that it is a real assessment tool because:
- It implies what assessors must do and requires their active involvement.
- It gives instructions to students on what they must do in the assessment, and these are used as criteria to establish a reliable result. In fact, by adding hyperlinks to more detailed instructions, it can even carry some of the instruction load.
The question of instruction is not too far away. In the past, computer-based instruction once favoured behaviourist instruction methods. The collaborative model of learning fostered in Moodle and Blackboard have created an alternative kind of education.
Good use of graphics, video, videoconferencing, and simulation has made computer-based instruction accessible to other kinds of learning styles and instructional methods. However, students still need a real person to ask their questions.
The task list
The assessment tool comprises a list of tasks that the students must do successfully. This system is suitable for on-job assessments in an actual workplace, especially if the workplace supervisor has a role in the assessment.
One of the most convenient features of a list of tasks is that they do not need to be secret. They are open and available to all students.
One of the variations is that assessments may be repeated, although there is usually a limit on how many times a student may be given any one summative assessment.
The most basic format
This format is assessed through the assessor's logon:
Unit(s) Assessment task Assessment result Satisfactory Unsatisfactory Tie your shoelaces ? □ Wash your face ? □ Comb your hair ? □ Get a handkerchief ? □
Name and location of hard-copy documents produced Comments to student Notes for review and moderation Assessment verified by Name* Time and date* *Electronically put in; identified by login and by the computer clock.
Variations
It would be easy to add hyperlinks to whatever else was necessary.
Unit(s) Assessment task Assessment result Satisfactory Unsatisfactory Tie your shoelaces
Click here for a full explanation of how to do this.? □ Wash your face
Click here for the relevant document template.? □ Comb your hair
Click here to try the simulation.? □ Get a handkerchief
Click here for the policy requirements? □
Name and location of hard-copy documents produced Comments to student Notes for review and validation Assessment verified by Name* Time and date* *Electronically put in; identified by login and by the computer clock.
Other variations
Self assessment
In this scenario, the student assesses him/herself to check whether they are ready for full assessment. It is not a requirement, but it can be greatly help students.
It is accessed through the student's log-on
Formative assessment
Assessors can give formative assessments to check whether students are on track for final assessment. It helps students know what they still need to work on.
Multiple assessors
Two people could confer on the assessment and give a final assessment. It would of course be difficult if the second assessor disagreed, but a database could not really resolve that problem.
It is accessed by the assessor's log-on.
Some tasks must be performed twice in different contexts. A few need to be done three times, and then it gets messier. (I'd suggest that a database-managed system might not do that wholly over the Internet.)
If both occasions were assessed by different people, the section of the form could look like this:
Assessment verified by first assessor Name* Time and date* Assessment verified by second assessor Name* Time and date* The next more complex step would be to include graded assessment. For the point of the illustration, it's enough to say that the possible results are A, B, C or D.
Grade awarded for unit: Reasons given for the grade Agreed by second assessor □ Yes □ No Notes for review and moderation:
First assessorNotes for review and moderation: Second assessor Assessment verified by first assessor Name* Time and date* Assessment verified by second assessor Name* Time and date*
Knowledge test
This really suits a traditional knowledge test done by interview and portfolio. It is assessed by the assessor's log-on.
Satisfactory answer Unsatisfactory answer Question 1 □ □ Question 2 □ □ Question 3 □ □ Question 4 □ □ Question 5 □ □ Question 6 □ □ Question 7 □ □ Question 8 □ □ Question 9 □ □ Question 10 □ □