Program review

Ross Woods, rev. '08, '12, 18

The purpose of this e-book is to enable you to conduct a program quality review and improvement of the whole organization or business unit. Continuous improvement is a required in the quality framework, and you probably already have ways of doing it. It is normally good practice to do it each year, but new programs need a much tighter review cycle.

A couple of principles to guide you:

  1. The first part of the whole process is to complile an issues register, which is a list of all kinds of problems and risks.
  2. Delegate as much work as you can. Involve other people to help them feel greater ownership of the results.
  3. Use at least some open-ended questions to catch issues that you might not have anticipated.
  4. If it is a large organization, divide it into business units.
  5. Before you start, inform your department heads so they don’t think you’re going behind their backs.

Decide how you will find out what people think

Your methods might include one or more of the following:

Questionnaires have the advantages of allowing you to carefully focus on the information you want, and collecting it from a potentially wide group of people. You can use a questionnaire by interview, mail, email, or handouts in a meeting.

For this purpose, you will probably get most benefit from open-ended questions. Respondents have freedom to give their own answers according to their own experience and perspective. They also give you freedom to ask follow-up questions. Writing the questions is fairly easy as long as they are clear and direct.

Be clear, and limit the number of questions to those that are most important. You should have no redundant questions although you may ask the same question in different ways if you need to check whether people answer consistently. All questionnaires need testing before using widely.

For program review, closed questions are not recommended. These are questions that only allow respondents to choose between certain alternatives.

Gather input and feedback

Gather input and feedback from all sources and compile a list of problems and risks of all kinds. You might have no option but to handle the most urgent issues immediately. You might find some small issues that you can quickly and easily resolve as you go, as long as you don’t get bogged down or distracted from the big picture. It can be easier to fix them straight away than to make them part of the whole review.

As you go, identify any changes in the operating context, such as marketplace, technology, regulations.

Compile an issues register

Collate all comments and feedback to find out what kinds of things you need to work on. Consider it a healthy sign if the review produces a long list of apparently serious problems. You have now identified areas of improvement.

Then go through the register, assessing each item, prioritizing it for action, and formulating solutions and improvements. Identify the gaps and suggest strategies to address them according to their priority. You might need to get approval for some solutions and improvements, e.g. major changes may need Board approval.

Develop a plan

At this stage, you need to oversee planning. Firm your issues analysis into a written quality improvement plan and consult stakeholders as you go.

It could be easy or complex. In the easy scenario, it might be as simple as working with your key people to fill in a review form regularly or taking a few minutes in a staff meeting. (But be careful; staff members are only one kind of stakeholder, so they may slant the findings in their favour.) It will proabably be much more complex in a larger or multi-site organization and you will probably need to delegate a considerable portion to other people. You should consider whether you need a change management strategy.

Similarly, making the changes is often simple fine-tuning in healthy organizations that are open to change, and need not be very stressful. But in sick organizations, it is major surgery with painful effects.

In your plan:

  1. Include a way of evaluating progress. This may be through using performance criteria, but it might also be risk factors or critical success factors.
    A critical success factor approach starts with the question: "Of all the problems we need to handle, which ones are critical to our success?" That is, what things must you get right? You then develop strategies for each factor and integrate them into a plan.
    How will you monitor implementation? It needs to be regular, but how often? Will it be in a staff meeting or a board meeting? You may need to prepare regular written or oral reports based on the plan and present them to staff or management meetings.
  2. Identify ways to build improvement into the way people work. A few organizations are always so busy "improving" that they are unstable and might be merely changing rather becoming better. But more likely, the trap is that it is just another form to fill in or a staff meeting item that nobody will act upon.
  3. Set a time when the whole plan will be reviewed.

Implement and monitor the plan

To avoid trivialization, get key people on side first. Then work with important priorities, delegate tasks, and follow people up to check that they’re doing things.

You will find that different people have different interests, and you will need to work with them all. For example, Jenni works in the office and is concerned over problems with computer records. Rocco is a practicum supervisor and couldn’t care less about records. But Rocco really wants something fixed in the field that Jenni just doesn’t deal with.

Later, when the whole plan is reviewed, you will have to revise it according to the feedback you get.