Problems and Purpose

Multi-criteria decision analysis is not exclusively a method of democratic innovation. However, it has been used in policy formulation within governments [1]. 21st Century Dialogue brought the method to community engagement through the multi-criteria analysis conference, outlined below:

"A Multi Criteria Analysis is a decision-aiding technique to analyse alternatives to complex problems using weighted triple bottom line criteria that are developed by all stakeholders. The end result is the 'best fit' option.

The process is systematic, structured, open and accountable. It engages all key stakeholders with their differing objectives. Both technical data and value judgements are used to reach a preference.

The MCA process has four key components:

1. A set of alternative options
2. A set of criteria for comparing the alternatives
3. Weighting to attach a measure of importance to each criteria
4. A method of ranking the alternatives based on how well they satisfy the criteria

The process involves four key steps:

1. Preparation and involvement of the community from the start;
2. An initial Workshop of all participants to determine the options and criteria;
3. An Expert Panel to oversee the quantitative data and to rate the qualitative data, with both sets of data being input to a specialised software package;
4. A second Workshop of all participants to weight the criteria according to their importance. Using both the quantitative and qualitative data, together with the value judgement weightings, the computer software determines the best options" [2].

Origins and Development

Know how and why this tool/technique was developed? Help us complete this section!

How it Works

There is no specific selection method for MCA but in their WA cases, 21st Century Dialogue employed three methods of recruiting participants with an equal split consisting of: stakeholders, a random sample, and people who responded to public notices of the conference. An MCA forum can have 60-100 participants. 

Central to the process is a panel of technical experts on the issue. The role of participants and experts is deliberately kept separate in the MCA process. Participants (the community) develop the options and criteria for them to be weighted against, whilst the expert panel does the weighting. Hartz-Karp [3] argues that this separation of roles is useful in avoiding conflict during deliberation about the proper place of expert vs. community viewpoints. 

Multi-criteria decision analysis (MCA) is a way for decision-makers (citizens or otherwise) to weigh up different options. Usually during deliberation, options are weighed up qualitatively, through discussion. MCA enables the weighing of options to be carried out in a more quantitative, rigorous manner. There are several stages.

1. The first step is to develop possible options or solutions to the issue at hand. This is done during a one or two day forum with participants from the community and stakeholder groups. During this forum, experts make presentations and answer questions from participants. Participants then use this information to identify possible policy options or solutions. They also develop a list of criteria against which the options will be weighted. The task is to simply define criteria because all options and criteria will be put forward. 

2. Following the forum, an expert panel meets and deliberates in order to weight the options against the criteria. Ultimately, the task of the panel is consider the relative impact of each option with relation to different criteria. For example, during Western Australia's freight network review, 21 possible options and 39 criteria were defined. These criteria were derived from the 'triple bottom line' criteria of economic, social, and environmental impacts. Thus, the role of the expert panel was to consider the impact of each option on each criterion and give it a numerical score. During this time, the panel is provided with any additional technical information that they require. The process can take several weeks to a few months depending on the complexity and number of options/criteria. 

3. When the expert panel has completed their analysis, a second forum is convened. The same participants from the first forum are invited to attend. During this session, participants review the decisions of the expert panel. Their task here is give different weighting to the criteria ie. rank the criteria in order of importance. Although deliberation in small groups take place, participants individually decide how much weight to give to each criteria.

4. The ultimate decision is made through computer analysis, once all the relevant information including the options and their relative weighting against the criteria, plus the weighting of criteria, have been fed into the computer. The final option presented is a 'best-fit option' [2]. The results are calculated and immediately shown to the forum participants to reflect upon. 

Analysis and Lessons Learned

The benefits of using MCA are the separation of roles of experts and community participants, and the facilitation of a rigorous, systematic decision-making process. It may be particularly useful for a complex issue where the weighing of options qualitatively may be more difficult. 

However, there are some limitations and challenges to consider. 21st Century Dialogue note that in cases, although participants accepted the final decision of the MCA, they also expressed frustration or dissatisfaction with the outcome. In the Albany Citizens' Jury, Hartz-Karp [4] notes that some participants felt that others had not been completely honest in their individual weightings, "biasing the final results in the direction they had preferred at the outset". This illustrates the disadvantage of private/individual choices vs. open collective deliberation to make decisions. The private element of the MCA resulted in a loss of trust between participants and ultimately, a lack of confidence in the final outcome. In the freight network MCA, although the outcome was accepted by participants, it was not widely accepted in the community and the issue continued to be a contentious and political issue.

MCA can also become a very labour and time intensive option. In the freight network review MCA, the extent of options and criteria submitted meant that the expert panel required additional time and resources to carry out their task. On the other hand, another MCA conference in WA on haulage routes yielded just two possible options for consideration, reducing the complexity and workload for the expert panel.

See Also

Western Australia's Freight Network Review

Albany Administration Centre Site Citizens' Jury

References

[1] Gov.uk (2009) Multi-criteria analysis: a manual [pdf], Department for communities and local government, available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/7612/1132618.pdf

[2] 21st Century Dialogue (2011) East-West Freight Route [online], available at: http://www.21stcenturydialogue.com/index.php?package=Initiatives&action=Link&file=east_west_route_to_freo_port.html [DEAD LINK]

[3] Hartz-Karp, J. (2005) Deliberation as a key to community engagement [pdf] in International Conference on Engaging Communities, August 14-17, Convention & Exhibition Centre, Brisbane, Australia, available at: https://publications.qld.gov.au/storage/f/2014-02-03T06%3A09%3A58.469Z/hartzkarp-janette-final.pdf [DEAD LINK]

[4] Hartz-Karp, J. (2007) Understanding deliberativeness: bridging theory and practice [pdf], International Journal of Public Participation, 1(2), available at https://www.researchgate.net/publication/265285114_Understanding_Deliberativeness_Bridging_Theory_and_Practice

External Links

UK Government - Multi-criteria analysis: a manual

National Resources Leadership Institute - Multi-criteria Decision Analysis

Notes