A simple card-sorting tool for qualitative research and inquiry, which can also be useful for planning and evaluation. A HCS can help make people’s informal and tacit knowledge more explicit and publicly available, contestable and usable.
Problems and Purpose
HCS is one of many types of card sorting methods (also known as pile sorting). Card sorting has been used in many contexts, from traditional ethnography to the modern day business of designing usable websites. In these contexts card sorting is typically used to elicit people’s mental models: the categories they use, what belongs to these categories, and how the categories relate to each other.
In many organisations people accumulate a lot of knowledge, but often it is tacit and informal in nature. As such. it is not so easily shared. Yet sharing that knowledge can make a difference, other people can make use of it, and they can help correct it and improve it. A HCS can help make people’s knowledge more explicit and publicly available, contestable and usable.
Origins and Development
This variant of card sorting is a tool developed in 1993, as part of PhD field work on organisational learning within an NGO in Dhaka, Bangladesh (the Christian Development Commission of Bangladesh (CCDB)). It has since been used as part of project design and project evaluation processes in Nigeria, Ghana, and the United Kingdom.
Participant Recruitment and Selection
There are two kinds of participants: an interviewer and respondents. Respondents may be a single individual or a small group - small enough to quickly come to agreement on their responses to the HCS task
How it Works: Process, Interaction, and Decision-Making
There are four main stages, the last two of which are optional
1. Listing the entities to be sorted. These can be people, organisations, locations, or events. The respondent should be expected to know something about them.
2. Sorting: This involves the reiteration of a question: “What do you think is the most significant difference between all these (entities). Please sort them into two groups of any size. Describe the difference between them and the difference it makes” The result is an annotated tree structure, a nested classification like the one shown above.
3. Choosing. The respondent can then be asked a second type of reiterated question, but here the format is more flexible. “Which of these two groups do you think has been more successful?” or “Which of these two groups have you spent more time with” And in each case asking a follow up question: “Why do you think so?”
4. Comparing. A succession of binary choices will generate an overall ranking, as seen above. Rankings of entities based on different choice questions can then be plotted and compared.
A detailed description of all steps can be found here: https://mande.co.uk/special-issues/hierarchical-card-sorting-hcs/#howdo
Influence, Outcomes, and Effects
Value can be obtained from HCS tree diagrams at two stages:(a) during the creation of the tree diagrams, and (b) through comparisons made between parts of the structure once it has been created.
During creation of a tree diagrams the main use is as a ethnographic tool: a means of understanding people’s view of the world. There are three types of use:
1.1 Identifying the distinctions that people see as important. This is evident in the contents of the differences reported. It is also evident in how early in the exercise they are reported, and how often they are reported (on different branches).
1.2 Identifying the limits to people’s knowledge: When respondents cannot identify differences between two or more entities the limits to their knowledge seem to have been reached. Knowing what people do not know about can be important, especially when they might be expected to, or claim to have, expertise in that area.
1.3 Identifying the direction of learning: It is also worth noting where there is more versus less differentiation of knowledge, visible respectively where where branches end up with a single case rather than multiple cases, which have not yet been differentiated. See Figure 2 above for an example.
2.Use after creation
After creation, the tree structure can be used as the basis for a series of judgements, about the future and the past. At each junction judgements can be made about the available binary choices i.e. between the two alterative branches. The process of soliciting and documenting these judgements can proceed in either of two directions: from trunk to leaves, or from leaves to trunk.
Some examples of questions that can be asked at each junction are:
- “How will your work in the next six months with this group be different, compared to this group?”
- “Which of these two groups do you plan to be spending more time with ?”,
- “Which of these two groups do you expect will present the most problems?”
2.2 Creating a Theory of Change with testable predictions
Each branch of a tree diagram can potentially be seen as a causal configuration, i.e.a set of conditions associated with an outcome seen in the entities that have been sorted . A complete set of branches can then be seen as a particular kind of Theory of Change, one that is notably different in at least two ways.
Firstly, because there will be multiple branches – it will be capable to representing equifinality i.e the reality that there are often multiple alternate means of reaching the same outcome. And asymmetrical causal processes i.e. that an outcome can be absent not because of the absence of its usual causes, but become of the presence of other influences.
Secondly, contra to more conventional representations, each segment in a branch is not a consecutive event, forming a chain of events. Rather, each segment is an additional kind of difference that is expected to make a difference to the outcome observable in the entity at the end of each branch.
2.3 Aggregating Most Significant Change (MSC) stories
In the uses described above the focus was on outcomes that are measurable in some way, at least using some form of ranking. But outcomes are often diverse in form and thus not easily measurable. For example in large scale decentralised programmes, and in programmes that emphasises peoples participation in the planning and management of programme resources. These are the circumstances where the Most Significant Change (MSC) technique is typically useful.
A tree structure generated by a HCS exercise provides a similar structure as a basis for summary by selection. The tree structure can be used like a tennis tournament structure. At the “branch ends” pairs of MSC stories are like entry level tennis players , who can be compared, and the most significant of these then promoted “up” to the next level. There they meet and are compared with another MSC story that has been similarly prompted from the level below. And so on, up the hierarchy.
Analysis and Lessons Learned
The most recent use of HCS was aby a consultancy firm, to carry out a stakeholder analysis in the initial stages of an evaluation. Key informants were interviewed using the HCS method, to identify what they saw as the most significant difference between listed stakeholders, and what difference those differences might make to the process, results and use of the proposed evaluation. The tree diagram that was produced highlighted 19 differences with different consequences. The results were assessed as being much more useful than more conventional stakeholder analysis methods such as the creation of an influence x interest plot.
There are also, like many forms of ethnographic inquiry , many areas where the process can experience problems. These are documented, along with proposed solutions, in this step by step guidance: https://mande.co.uk/special-issues/hierarchical-card-sorting-hcs/#howdo
Some extra resources on card/pile sorting and related methods:
- Harloff, J., & Coxon, A. P. M. (2007). How To Sort; A short guide on sorting investigations.
- Borgatti, S. P. (Ed.). (1999). Elicitation Techniques for Cultural Domain Analysis. In Designing and Conducting Ethnographic Research (Ethnographer’s Toolkit).
- Coxon, A. P. M. (1999). Sorting data: Collection and analysis. SAGE.
- Gladwin, C. H. (1989). Ethnographic decision tree modeling. Sage.
To do any type of sorting exercise a list of cards to be sorted is necessary. Such lists can be identified/generated using unstructured interviews, or by using more structured ethnographic methods such as freelisting. Some suggested reading:
- Borgatti, S. (1998). Elicitation Techniques for Cultural Domain Analysis. In The Ethnographer’s Toolkit (Vol. 3). Altimira Press.
- Quinlan, M. (2019). The Freelisting Method (pp. 1431–1446). https://doi.org/10.1007/978-981-10-2779-6_12-2
Davies, R. J. (2021) Hierarchical Card Sorting (HCS) aka Most Significant Difference (MSD), on the MandE NEWS website, https://mande.co.uk/special-issues/hierarchical-card-sorting-hcs/