Evaluation is a planned process of assessment of the extent to which the work undertaken delivered its intended objectives. Metrics refers to the specific data or measures used to assess the outcome. Whether a specific metric is a valid indicator of an will depend on the projects objectives.
The OD Practitioner will;
- Help the client to agree the general areas of evaluation and the corresponding relevant metrics during an inception meeting
- Build the evaluation areas and metrics into the OD programme report
- Set up a tracking mechanism from the beginning of the programme – so that evaluative data can be collected throughout the project life and be used to adjust the project approach if required.
- Involve the Programme Team to do evaluative data gathering to increase programme ownership. Use the evaluative process to reinforce the development agenda with energy and commitment
- Present coherent to the organisation
- Help the Organisation to determine what actions to take to adjust the outcome of the programme interventions – reinforce what works, address what does not work, design further interventions etc.
Specific Programme Questions for Organisations to consider;
- What do you want to get out of the Programme?
- What would success look like to you?
- If we were to ask other stakeholders about the desired outcomes they want from this programme, what would they be?
- What type of metrics will satisfy the key decision makers in this programme?
Examples of Evaluation and Corresponding Metrics
|
Type of Intervention
|
Level of Evaluation
|
Explanation
|
Example of metrics and methods
|
| Workshops |
- Reaction, satisfaction and planned action
- Short-term one off intervention
|
- Measure participants reaction to the focus group
- Measure stakeholders satisfaction with the programme
- Measure the immediate response to new practices
|
- Post event evaluation form
- Telephone interview of random sample of participants
- Survey six weeks after event
|
| Mentoring |
- Learning
- Short or Medium Terms
|
- Measure skills, knowledge, or attitude changes related to the programme
- Measure rate of implementation of new learning
|
- Before and after skills/knowledge test
- Two-party joint evaluation of the effectiveness of mentoring
- Report from supervisors on impact of the scheme
- Direct observation of performance
- Participants self-assessment of impact
- Rating of actual performance improvement by customers, partners and peers
|
| Joint working |
- Application and implementation
- Short, medium and long-term intervention, but tend to be more complex
|
- Measures changes in behaviour on the job
- Specific application and implementation of what they have learnt from Coaching
|
- Line leaders’ direct observation of the rate of improvement.
- Self reporting on new experimental behaviour for join working
|
Involve whole system in diagnostic intervention
|
- Business Impact
- Medium and longer term interventions and almost always complex projects
|
- Ownership rate of the diagnostic plan
- Measure project impact on business
|
- Number of strategic priorities delivered and on what timescale compared to previous two years.
- Pre/post customer complaint record or satisfaction index
- Cost of repairing customer service mistakes compared to previous two years
- Employee satisfaction rating
|
Evaluating the Progress of a Project
The purpose of this is to reflect on how the team is doing and decide what modifications that are required to ensure the team’s success
Sample Session Outline;
|
Session
|
Process
|
|
Opening
|
- We are at a critical point in our intervention. We need to reflect on progress to date. I expect that each of you have reviewed the overall plan and the reports from each of the project groups. We want to reflect on what has been accomplished this month and decide if we need to adapt the plan to keep the project on track and on schedule
|
|
Objective Questions
|
- As you read these reports, what statements or points caught your attention?
- Where were the reports clear?
- Where are they not clear?
- As you think of our work as a team, what other information do need to share?
- As you compare these reports to our project plan, where do we appear to be ahead of schedule?
- Where do we seem to be behind schedule?
|
|
Reflective Questions
|
- What surprises did you find in the reports?
- Where has the work gone easily?
- Where has it been more difficult than expected?
- Where have we run into unexpected issues or challenges?
|
|
Interpretive Questions
|
- What appear to be the key issues or key problem areas?
- Where will we need extra help?
- What kind of help will we need?
- What questions do we need to work through as a whole group?
- What will it take to keep this project on track or get it back on track?
|
|
Decisional Questions
|
- What changes are we recommending to the plan?
- What changes are we making to the way we need to operate?
- What are the next steps?
- Who needs to do them?
|
|
Closing
|
- This has been very helpful to us all in getting the big picture and seeing where we need to move next
|
