Being audited

It's coming

If you're in an accredited school or college of any kind, the accreditor will audit your college for compliance with its quality framework. Less specifically, any kind of organization can be subject to a quality audit, especially an ISO9000 or AS9001 audit, which is a generic standard for any kind of organization.

Audits are onsite checks of compliance with defined standards. They mostly comprise interviews and checking documents of various kinds, and comparing the two. All audits are documented, and audits always have a defined scope. Remember, an audit is of the organization's compliance with the standards. It is not an audit of an individual.

This means that an auditor expects your organisation to have written policies and procedures, and records of what you have done. They can ask you and your staff whether you know, understand, and apply these requirements. You can be asked for relevant documents.

Stages of audit

Audits follow a specified procedure and set of standards. They should follow these six stages:

1. Initiate the audit
2. Appoint a head auditor
3. Establish an audit plan
4. Hold an opening meeting with senior management of the organization being audited
5. Conduct the audit
6. Hold a closing meeting and report

What will be audited

You should have been informed beforehand that you will be audited, but this might not necessarily have happened. Auditors have a right to follow up spontaneously anything necessary to the audit, so you cannot predict specifically what they will audit.

At the beginning of the audit, help the auditor understand your organization. It is a good idea to give the auditor a snapshot of your scope of delivery, number of students, modes of instruction, staffing, facilities, client groups, special features, etc.

As a trainer and assessor, you could be involved when the auditor checks the following:

• Whether you have done a staff induction course that includes an organizational chart.
• Your job description or duty statement
• Obtaining student feedback
• Continuous improvement of the program

You are required to comply with legislation and regulatory requirements on matters such as workplace health and safety/ duty of care, workplace harassment, victimization and bullying, anti-discrimination (including equal opportunity, racial vilification, disability discrimination, and vocational education and training. You might also be required to provide students with this kind of information.

You are required to:

• give the auditor access to all necessary materials for your function
• keep up-to date records of your experience and qualifications, and
• have version controls of written materials.
• have the appropriate qualifications in assessment and workplace training (normally the Certificate IV) and have these on file.
• safeguard confidential information in your care,
• provide students with access to their records, and
• take opportunities for professional development.

You are auditable for planning learning and assessment strategies, and for monitoring your performance and assessments. This may be by having person or a committee that approves your assessment plans or collates your feedback.

If you're in an accredited college and advise potential applicants and new students, you can be required to:

• recognize qualifications and statements of attainment from other accredited colleges,
• encourage Recognition of Prior Learning by offering it to prospective students
• maintain principles of equitable access
• provide students with timely and appropriate advice to help them achieve their desired outcomes, and
• ensure that a fairly large amount of specifc information is provided to applicants.

How to prepare

The best way you can prepare for an external audit is to go through your systems beforehand and check they are all up to speed, and then go through a rigorous internal audit based on the same standards. The tougher the audit at that stage, the easier the external audit will be.

Start by doing a an organizational self-evaluation:

• Check the audit requirements.
• Complete a self-evaluation report within your organization that reviews practice against the standards.
• Get feedback from clients and other stakeholders using focus groups, questionnaires or interviews with clients.
• Conduct an internal audit of your records.
• Do all necessary documentation.

Some organizations use an external consultant to prepare for an audit. From an outsider's viewpoint, the consultant can see things that an insider cannot. However, the audit is of the organization's compliance and the interaction is between the auditor and the organization, which does not include the consultant. Although the consultant may attend the audit to support the organization, he/she may not respond to the auditor on the organization's behalf or take part in any discussions on the conduct, progress or findings of the audit.

The next stage is to prepare and implement a quality improvement plan. If you don't do it yourself, you must facilitate it. First, assess your organization's performance against the standards and identify gaps. Then develop strategies to address the gaps. If necessary, consult relevant stakeholders to help you.

Then implement the plan. Develop an approach to make the quality improvements and integrate them into your work systems. It must include performance criteria. Implement the plan and monitor it regularly. As you go, do any reporting to any relevant parties (e.g. your supervisor and staff) and revise the approach according to the feedback you get.

When you have done your homework, you can prepare for the actual audit.

1. Designate a principal auditee. This will normally be the senior person in the department being audited. That person will do most of the preparation for an external audit. He/she
• helps keep the audit on schedule
• makes staff available at planned times
• liaises between the auditor and the organization's staff.
2. Inform all stakeholders who need to know about the audit
3. Make sure all staff are familiar with all processes and how they work, and are already implementing any new processes.
4. Check the documentation in your application
5. Determine what evidence you will need for the audit.
6. Prepare files and records and any other documentation for examination. For example, you need to have all documentation arranged where you can locate it at a moment's notice. As you will be nervous (nobody likes being audited), it creates a very bad impression if you are flustered and can't find things easily.
7. Decide which staff and board members could provide supporting evidence and make sure they will be available during the audit. Schedule any necessary interviews with staff and any other stakeholders (e.g. service providers).
8. Allocate the auditor a desk or workspace. The auditor will need a place for document analysis and interviews.
9. Inform the auditor of any special access arrangements. These may be safety clothing, security or parking arrangements.

Brief people thoroughly. Audits are stressful, so put your staff at ease with the audit process. Let them know what could be expected of them. A more complete explanation is provided in another chapter, but it is usually:

• providing honest answers to straight-forward questions.
• being observed teaching
• demonstrating how they do things and explaining why they do it the way they do.

__________
Ref. Preparing for your Australian Quality Training Framework (AQTF) Audit Version 2, Training Accreditation Council, Perth WA. July 2004.
Also based on Manage an accreditation process (CHCQM601A)

New programs

An audit of a new program is different from an audit of an existing program.

The point is that the auditee must show that they have everything already in place and is ready to run the entire program to standards. This is different from starting to get ready; the preparation must already be done. Besides, you cannot get recognition for something that you might want to run sometime in the future; you need to be ready when being audited. This means the RTO should do the following in their planning.

Be ready with:

• Filled-in training plan forms
• Prepared assessment plans and forms
• Demonstrated need for the program and evidence that students will be forthcoming
• Letters (or other documentation) of industry consultation expressing agreement on the training and assessment approaches.

Be prepared to explain how the program will work to the validator and the difficulties that you anticipate. If the program is already running in unaccredited form, then you do not have have this burden. You only need to explain how it works and the changes that accreditation will make.

For your staff, you will need:

• Certified copies of relevant staff qualifications and Curriculum Veti
• Evidence of meeting Certificate IV in Training and Assessment requirements
• Evidence of keeping up your skills in both your vocation or profession, and in competency based training and assessment.

Be ready to show the auditor your facilities and resources:

• Rooms and equipment
• Access to any other facilities necessary for the course, e.g. practicum institutions, libraries
• Teaching and assessment materials, e.g. texts, training manuals, job descriptions, etc.

Being audited

In the entry meeting, the auditor will discuss the audit plan with you and generally lead the whole process thereafter.

Auditors have the authority to actively explore anything that they suspect to be non-compliant. They do not just passively accept whatever you show them, because a document is not in itself necessary conclusive evidence of compliance—people write all sorts of things hoping that auditors will believe them. Auditors will also often ask what you actually do.

Auditors are free to interview people, and want to see:

• what you actually do (or plan to do)
• how you have documented it (or plan to document it), and
• your understanding of the requirements.

You can also be interviewed on your understanding of any policies that are relevant to you. So what do you say if you can't remember the answer?

• You should not say "I don't know."
• You should say "I don't know but I know where to find the answer straight away". (For example, you might go to the policies or legislation areas of the website.)

Small non-compliances will need to be checked until compliance is attained. If you have substantial non-compliances, your whole program may need to be re-audited.

Unfair judgments aside, don't worry too much if the auditor finds non-compliances. Unless they are major disasters, you should have an opportunity to provide extra evidence to fix it. In fact, it's part of your learning curve.

If you are the principal auditee, it is likely that the auditor will ask you more questions than anybody else. However, the auditor cannot rely only on your answers, so be sure to avoid monopolizing the audit.

Good and bad auditors

Good auditors:

1. make an effort to understand the particular dynamics of the organisation
2. will often identify program strengths
3. ask you for suggestions on how you want to improve
4. make the process as beneficial as possible for the auditee
5. leave the auditee with a feeling that the process has been helpful
6. differentiate between four occurrences that can all look like non-compliances:
• possible improvement of program quality
• identified risk
• clerical errors
• possible better documentation
• actual non-compliance.

Poor auditors go on a fault-finding mission, feeling they have failed if they don't identify faults.

Auditor mistakes

The most common problems at present are volume of learning and amount of training. "Volume of learning" means the total period of time of the course. "Amount of training" refers to the amount of time give to providing instruction during that time. While the intent of these principles is honorable, any determination of non-compliance is a judgement call, and not a direct application of criteria. Consequently, a statement of non-compliance is subject to dispute.

Handling the mistakes of less experienced auditors is one of the more difficult problems in applying a quality framework:

• The auditor may not have fully informed the auditees of his/her findings during the audit or final interview. If this happens, the first the auditees might hear of alleged shortcomings is when they receive the auditor's written report, which the accreditor will send after the validation.
• Accreditation authorities try to avoid admitting that their auditors made mistakes, apparently for legal and administrative reasons. There are no problems, just "opportunities for improvement" or "review of current practices."
• Some auditors seem to feel very responsible to be sole safeguards of standards, and feel a greater pressure to find faults.
• Less experienced auditors have more difficulty seeing diverse and creative ways of meeting requirements. The quality framework then means whatever they will tick a box for.
• Some auditors have appeared to work on the basis: "If in doubt, conclude that the RTO is noncompliant." They seem to go in for the kill when they have doubts or suspicions about a program, even disregarding evidence of compliance. A recommendation for de-registration is the probable outcome. The accreditor will then send a letter asking the RTO to show just cause within a certain time why it should not be deregistered.

There is a double bind for the auditee. The auditee is responsible to show the validator what he/she needs to see, but the validator is basically in control of the process.

A speedy, amicable solution

A speedy, amicable solution is best of all. Auditors are generally trying to do a good job. So don't get angry or vent frustration; increased tension will further derail the audit.

Accreditors once didn't really want to de-register RTOs that are honestly trying to comply with the rules, and they got more kudos by showing that you are doing well. In fact, a good auditor won't make serious threats about problems that are easily solved. They usually give you a chance to submit more evidence and fix small problems.

Check the auditor didn't miss some evidence

The first step, simplest of all, is to check that the auditor didn't miss some evidence. Simply point it out and ask for their response. Here are several scenarios where it's very easy to miss evidence:

• In desk-top audits with lots of paperwork, diligent auditors easily miss things even if they are clearly laid out.
• You might offer evidence for one point that is also relevant to another point later on. However, both you and the auditor forget that evidence when auditing the later point.
• The auditor might misinterpret industry-specific terminology, especially if it has another common meaning.

At time of revision, the discussion in the RTO network gives the impression that auditors are now trying to de-register as many RTOs as possible, sometimes for frivous reasons. Certainly, the current VET Act gives the ASQA Comissioner Stalinesque powers to de-register RTOs.

Ask specifically what is non-compliant

If an auditor alleges that you have a non-compliance, one way to resolve it quickly and easily is to ask the auditor specifically what is non-compliant. (Under auditing guidelines, auditors are obliged to make sure you understand the audit findings.) The auditor then has to:

• be very clear and specific in saying what is non-compliant. Even when external auditors are not permitted to give you actual advice, this is the next best thing. If they got the judgment right, they must be willing to do this. Good auditors do it well.
• withdraw the allegation of non-compliance because he/she cannot really specify what is non-compliant. In the past, it was surprising how often this happened.
• bluff with a vague answer. Your first response is to politely ask more purposeful questions. If this is inconclusive, you have cause for complaint and their allegation could be overturned simply for lack of anything specifically noncompliant.

Fix the problem straight away

Unless the auditors finds serious, systemic non-compliance, you will normally have opportunity to provide extra documentation to cover any gaps. Even if you're not wrong, perhaps it's not worth a disagreement. It might be simpler to provide evidence of changed procedures and implementation.

Complaints and appeals

If a simple solution hasn't resolved the problem, you still have some not-so-drastic avenues open, but you have little cause for optimisim.

Challenge the audit in writing

If auditor mistakes have unjustly resulted in an unfavorable outcome, the RTO could provide a polite, objective written response to the auditor's report stating its reasons. This is probably a waste of time given ASQA's current approach to compliance. ASQA will probably dismiss it by return mail.

In the past, accreditors would review alleged non-compliances and sometimes suddenly find that the RTO is much more compliant than the auditor had reported.

Meet with the accreditor

RTOs could once meet with senior accreditor staff by appointment:

It was good advice to take a friendly, polite, reasonable approach and it would get a helpful response. It was possible the accreditor was right and the RTO was wrong, but at least Australian government departments were generally very helpful to people who honestly did their best to abide by the rules.

They could not bend the rules or procedures, or speak negatively of their own people or of government policy. Like now, they probably knew the political background to what is going on better than you, but were not always able to disclose it.

They would probably explain matters well, answer any difficult questions that are fair, and give you good advice. It was unlikely that they will fob you off but RTOs did not need to accept unconvincing or superficial answers.

RTOs could thank them afterwards and should put discussion results in writing. (It was also good practice to email them a copy, as long as your notes are accurate and objective.) A good solution was in their interests too.

ASQA is now centrally contolled and heavy-handed. The people who answer the phones or emails cannot help. Sometimes they do not even understand what you have said or written. They are not authorised to say anything outside their given procedures, and have little or no knowledge of how to run an RTO.

Complaints

The ASQA complaints process is a waste of time; it will probably be dismissed by return mail.

Freedom of information and the Privacy Act

Be aware that freedom of information legislation and the Privacy Act gives you the right to view any records held by the government on your case. Consequently, public servants are usually very careful about what they put in writing.

Ombudsman

The federal Ombudsman offers an interesting service. Most complaints to the ombudman are rejected by returen mail, mainly because they lack substance. Consequently, a complaint needs to have some real evidence of a case worth pursuing.

The Ombudsman has the power to demand your file at short notice, and to investigate the matter independently. Although the Ombudsman can't reverse any decision, public servants really, really don't like being being investigated by the Ombudsman.

Appeals

If the validator made a mistake and put the RTO in a difficult position that cannot easily be resolved, consider vigorously pursuing an appeal. This is now done through the Administrative Appeals Tribunal, a kind of court case. The VET Act gives ASQA so much power that a successful appeal outcome is unlikely.

The first stage is mediation, but you will only be allowed to talk to an ASQA lawyer, who can discuss procedures and take messages, but cannot actually discuss the case. You will be unable to talk directly to any ASQA decision-makers.

to the is so CT mbiguities in the AQTF and validation processes have reportedly generated many complaints in the past.

Make sure that you have a very strong case, and that you have expressed it clearly and dispassionately in writing.

Check both the accreditor literature and legislation; they will inform you of various avenues.

You should be careful not to prejudice your case (or slow it down) by using one avenue when the other is correct. In more extreme cases, however, it is possible to use multiple avenues simultaneously.

Parliament

Parliament will probably get you fairly slow action, it at all. Some cases are reported handled by junior bureaucrats who dismiss comments by return mail, often without the Minister's knowledge. If it progresses, it will normally go by formal letter to the Minister, who will write to the accreditor.

The Minister's letter to the public servant is called a Ministerial, and public servants detest them as they are normally very urgent. Answers must not be politically embarrassing and must be honest.

But the system is not necessarily weighted against public servants. If a public servant is the cause of the problem but is not corrupt, it is quite likely that he/she will write the reply to the minister and cover their back. Besides, public servants can invoke a wall of policies for their protection.

Outcomes

In the meantime, the registration agency would consider an unfavorable audit outcome. They could:

• issue fines, especially if a non-compliance was serious or preventable
• give the RTO an extension of time to provide extra evidence of compliance
• cancel registration of scope for an industry area or a qualification (but not the whole RTO)
• appoint a re-audit
• suspend registration pending further action. At one stage the normal practice was to allow enrolled students to continue but to disallow the RTO from advertising, admitting new students, and issuing qualifications

In times past, accreditors saw full cancellation of registration as a drastic step and not lightly taken. No longer. It now de-registers RTOs for marginal reasons.

Litigation

Litigation is a last resort, and you must get legal advice before pursuing it. Litigation should only be used when you can clearly prove your case, and your particular case will probably have specific features that this general discussion cannot cover.

The ASQA Commissioner has power to can issue whatever rulings he/she likes and enforce them as law. The commissioner has not used this power often, and does so only with consultation. Consequently some ASQA rules have no legal basis in the VET Act, Regulations, parilamentary instruments, or Commissioner's determinations.

It is unlikely that you will be able to talk directly to any decision-makers outside legal proceedings; they will only let you speak only to a lower-level person. Then, they may decide to drop their actions against you only on condition that drop your allegations, indemnify them against all damages, and keep it all secret. That is, you might find yourself dealing with bullies who force you to either settle on their conditions or go to court.

Some lawyers lodge a letter of intent to sue on all possible defendants, for example, the auditor, the auditor's company (if it is a separate entity from the auditor), and the accreditor.

Your petition should include a guarantee of non-prejudicial treatment in the future. In some cases, it is advisable to petition that the case be from your records. In jurisdictions that have laws protecting government records from change or deletion, it might be unrealistic to ask for expungement.

Liable for what?

Auditors cannot be held liable for:

• the quality of a program they audit, or
• audits that use valid sampling techniques but find no problems in the sample.

Auditors could be held liable for dishonesty. Too many auditors have ignored legitimate evidence of compliance, or made up a rule that differs from the standard against which they are auditing.

Negligence is also fairly probable, because it's easy to miss things that in hindsight should have been investigated. However, you might be required to prove that the auditor had not exercised due diligence. (That is, if the auditor had exercised due diligence but still missed something, it could be no more than an innocent error.)

Although unlikely, auditors could also be guilty of collusion, breach of confidentiality, and conflict of interest. When audited information is commercial in confidence, the sums of money at risk may be very substantial.

Auditors could be held liable for errors that lead to incorrect summative decisions such as recommendations for de-registration or cancellation of scope in an industry area.

Burden of proof

The burden of proof in civil cases is weight of probablility. You don't have to prove that they are guilty beyond reasonable dougbt; you just have to have a stronger argument than they do. Still, not as easy as it sounds.

If the error occurred during the audit, you would probably have to demonstrate all six points below, preferably by documentation rather than recollection only:

1. The auditor error resulted from either the auditor or the accreditor:
1. being negligent (failing to ask for, consider or check relevant evidence),
2. being dishonest (e.g. refusing to accept valid evidence of compliance, fabricating evidence of non-compliance),
3. failing to comply with required procedures, or
4. being inappropriately qualified. This applies if the crux of the case is auditor comment on subject matter in which they have no expertise. For example, he/she had only ever taught Cert II in Floristry but recommended your de-registration because he/she disagreed with your assumptions in an engineering research course.
2. The RTO or its employees suffered serious detriment that you can quantify in financial terms, such as financial loss or adverse health effects. (Emotional distress is less recognized for compensations that it once was, but it may be a factor.)
3. The detriment is a direct consequence of the auditor error.
4. You upheld your responsibilities as auditee. For example, you cooperated with the auditor and disclosed relevant information, etc. (That is, you did not substantially contribute to the auditor error.)
5. You promptly took steps to resolve the matter. For example, you lodged extra evidence, wrote or visited seeking advice or clarification from the accreditor, or lodged a complaint, appeal, or objection.
6. The complaint or appeal procedure was defective. For example:
1. it was not prompt enough to prevent any detriment to you (the most likely scenario),
2. it was negligently applied by the accreditor (e.g. they lost your file, gave you incorrect information because they didn't bother to check it, or postponed adjudication by failing to recognize probable consequences),
3. it was dishonestly applied by the accreditor (e.g. they made claims that were demonstrably untrue),
4. it was recklessly applied by the accreditor (e.g. they did things without checking whether or not they were correct),
5. it did not comply with pre-existing procedures or standards, such as the requirements for accreditors.

In the past, accreditors seemed anxious to avoid legal proceedings for several reasons:

• It was a political embarrassment to the Minister of the day.
• They faced political consequences, especially if the matter goes to the press.
• Their confidentiality requirements weakened their position; they cannot disclose any audit information to the public.
• They could not claim that a particular interpretation of the quality standard was legally wrong because they had no public, transparent system of interpreting it with which RTOs were required to comply.
• Auditors used a moderation system in which auditors compare their judgments to a given set of evidence, but it didn't generate any concrete interpretations that can be used to guide further interpretation.
• RTOs could argue for a prima facie interpretation and only had to prove that you were compliant.

The situation has now changed. Politicians no longer see a political disadvantage in actively de-registering RTOs, and have prosecuted the worst cases. Most de-registrations have not made court cases, even when ASQA had a weak case.

Auditors still use a moderation system, and the results are available on Freedom of Information legislation. However, it is is simply of a system of checking that they have a sound interpretation of the standards. It does not have the strength of legislation; so RTOs still have the avenue, however unlikely, of arguing for an interpretation to prove that they are compliant.