| The most significant change (MSC)
technique is a form of participatory monitoring and evaluation. It
is participatory because many project stakeholders are involved both
in deciding the sorts of change to be recorded and in analysing the
data. It is a form of monitoring because it occurs throughout the
programme cycle and provides information to help people manage the
program. It contributes to evaluation because it provides data on
impact and outcomes that can be used to help assess the performance
of the programme as a whole.
Description
Essentially, the process involves the collection of significant
change (SC) stories emanating from the field level, and the systematic
selection of the most significant of these stories by panels of
designated stakeholders or staff. The designated staff and stakeholders
are initially involved by 'searching' for project impact. Once changes
have been captured, various people sit down together, read the stories
aloud and have regular and often in-depth discussions about the
value of these reported changes. When the technique is implemented
successfully, whole teams of people begin to focus their attention
on programme impact.
MSC has had several names since it was conceived with each emphasising
a different aspect, e.g. 'Monitoring-without-indicators': MSC does
not make use of pre-defined indicators, especially ones that have
to be counted and measured; or the 'story approach': The answers
to the central question about change are often in the form of stories
of who did what, when and why - and the reasons why the event was
important.
Overview of implementation steps
MSC is an emerging technique, and many adaptations have already
been made. These are discussed in Davies and Dart (2005). In sum,
there are 10 steps:
- How to start and raise interest
- Defining the domains of change
- Defining the reporting period
- Collecting SC stories
- Selecting the most significant of the stories
- Feeding back the results of the selection process
- Verification of stories
- Quantification
- Secondary analysis and meta-monitoring
- Revising the system
Description of process
The first step in MSC generally involves introducing a range
of stakeholders to MSC and fostering interest and commitment to
participate. The next step is to identify the domains of change
to be monitored. This involves selected stakeholders identifying
broad domains - for example, 'changes in people's lives' - that
are not precisely defined like performance indicators, but are deliberately
left loose, to be defined by the actual users. The third step is
to decide how frequently to monitor changes taking place in these
domains.
SC stories are collected from those most directly involved, such
as participants and field staff. The stories are collected by asking
a simple question such as: 'During the last month, in your opinion,
what was the most significant change that took place for participants
in the program?' It is initially up to respondents to allocate their
stories to a domain category. In addition to this, respondents are
encouraged to report why they consider a particular change to be
the most significant one.
The stories are then analysed and filtered up through the levels
of authority typically found within an organisation or programme.
Each level of the hierarchy reviews a series of stories sent to
them by the level below and selects the single most significant
account of change within each of the domains. Each group then sends
the selected stories up to the next level of the programme hierarchy,
and the number of stories is whittled down through a systematic
and transparent process. Every time stories are selected, the criteria
used to select them are recorded and fed back to all interested
stakeholders, so that each subsequent round of story collection
and selection is informed by feedback from previous rounds. The
organsiation is effectively recording and adjusting the direction
of its attention - and the criteria it uses for valuing the events
it sees there.
After this process has been used for some time, such as a year,
a document is produced with all stories selected at the uppermost
organisational level over that period in each domain of change.
The stories are accompanied by the reasons the stories were selected.
The programme funders are asked to assess the stories in this document
and select those that best represent the sort of outcomes they wish
to fund. They are also asked to document the reasons for their choice.
This information is fed back to project managers.
The selected stories can then be verified by visiting the sites
where the described events took place. The purpose of this is two-fold:
to check that stories have been reported accurately and honestly,
and to provide an opportunity to gather more detailed information
about events seen as especially significant. If conducted some time
after the event, a visit also offers a chance to see what has happened
since the event was first documented.
The next step is quantification, which can take place at two stages.
When an account of change is first described, it is possible to
include quantitative information as well as qualitative information.
It is also possible to quantify the extent to which the most significant
changes identified in one location have taken place in other locations
within a specific period. The next step after quantification is
monitoring the monitoring system itself, which can include looking
at who participated and how they affected the contents, and analysing
how often different types of changes are reported. The final step
is to revise the design of the MSC process to take into account
what has been learned as a direct result of using it and from analysing
its use.
In sum, the kernel of the MSC process is a question along the lines
of: 'Looking back over the last month, what do you think was the
most significant change in [particular domain of change]?' A similar
question is posed when the answers to the first question are examined
by another group of participants: 'From among all these significant
changes, what do you think was the most significant change of all?'
Source
Further resource
- View this as pdf
(
59kb)
Back to Successful Communications tools
|
