What is made possible?

The Deliberative Delphi (Citizens Delphi) is a qualitative co-creation and participation process for stakeholder surveys and expert consultations. Analysis methods specially adapted to target setting, action planning and solution sketching are used in the evaluation. Characteristic of the variant is, on the one hand, the use of mini-workshops (“remote phase”) as the primary means of communication, and on the other hand, the use of debate and role-play formats.

As with every insights process, results of a deliberative Delphi are findings. A cognition can take the form of a concrete individual reference to the solution of a problem or a task or it can consist of a comprehensive picture that shows the solution components in context. With the focus on knowledge, a Deliberative Delphi differs significantly from opinion polls, where the amount and proportion of votes cast are relevant for decision-making. On the other hand, the Deliberative Delphi also differs from participation formats in which groups of participants work out solutions together and by consensus. In a Deliberative Delphi, instead, participants contribute pieces of a puzzle to solve a problem or a problem. Participation takes place here in the mode of participation or contribution. For those involved, but also for outsiders, it is transparent to understand which contributions the final findings are based on.

How it works: Brief instructions in six modules

The selection or recruitment of participants precedes the actual process. These constitute the project group. Ideally, this should be a group of 15 to 25 people.

1. Kick-off

The process starts with a face-to-face event in the form of a creative workshop. During the workshop, the project group familiarizes itself with the topic and sets priorities for the subsequent remote phase.

2. Remote phase

During the subsequent multi-week remote phase, the moderation team conducts repeated mini-workshops with members of the project group. As a basis for discussion in the mini-workshops, the moderators create editorial content and process research assignments from the participants. Debate and role-play formats are used, such as the presentation of information in a positioned form, i.e. as pro and contra contributions. The aim of these formats is above all to address the participants emotionally and thus to motivate them more in terms of content. Another goal is debiasing, that is, the avoidance or reduction of cognitive distortions such as the myside bias. This leads to the fact that, above all, information is perceived and processed, which confirms one's own preconceived opinion.

In addition to two moderators: inside (or one moderator: in and one representative: in the participating instance), two to four, a maximum of five guests take part in each mini-workshop. The workshop lasts one hour.

In terms of content, the mini-workshops are designed as key question interviews. The same key questions are used for every workshop on a given topic. The key questions take the participation question from online participation, vary the wording and go into detail. The workshops are moderated in such a way that, overall, they have more of the character of a conversation and a mutual exchange than that of an interview.

The participants' contributions are recorded as transcripts. This is done in the form of stylized, i.e. modeled, quotations that get to the heart of the content of an article as closely as possible. The quotations are not identified by name.

Workshop protocols summarize the results from the entire project group. Before a new round of workshops starts, the members of the project group receive the interim results for inspection.

A quantitative survey of the project group can also take place at the beginning or at the end of each workshop round - either in the mini-workshops or as a written survey.

3. Analysis, first step: Isolation of key messages

In the course of the analysis, the moderators isolate individual key statements from the participants' answers. Key messages are traditionally up to 200 characters long (including spaces). This is done with the help of the digital Insights platform or a relevant software for qualitative content analysis (see below, tips and stumbling blocks). Ideally, the corresponding software should make it possible not only to mark passages within a text (and to assign categories, see stept “Clustering”), but also to comment on the marked passage with a freely selectable text (which then constitutes the core message). It is important that at the end of the day it is possible to be able to call up the corresponding passage in the original article from the individual core statement.

4. Analysis, second step: clustering

The key messages are now clustered inside by the moderators, that is, arranged in groups. The grouping is mostly done inductively: Instead of starting from a given scheme, the categories are developed and adapted on the go. There are no generally applicable guidelines for clustering. It is often useful to use relevant problem structuring categories (such as “problem” and “solution”). In most cases, however, it makes sense to develop the categories from the content of the topic.

As with the formation of the core statements, clustering is also carried out with software support. For more sophisticated analyzes, ideally Insights software or common tools for qualitative content analysis are used. In the case of extensive consultations, it is advisable to use software that supports the analysis through keyword searches or artificial intelligence processes.

5. Analysis, third step: Forming insights

The results of the clustering are the basis for the final findings. In the simplest case, a finding is a summary of participant contributions for a specific category. However, the moderator should endeavor to clearly differentiate findings from the reproduction of mere opinions and not necessarily treat opinions expressed in the consultation as facts. If opinions that are clearly recognizable as such are involved, then these can, for example, become a realization that they are interpreted as information about different needs and wishes within a target group or are read as a summary of which ideas or arguments are at least prima facie for worth considering. Findings can also consist in the fact that participants contribute detailed knowledge in their contributions that was not previously known or not sufficiently taken into account beforehand.

As with the creation of the categories, when formulating findings it is also advisable to work with relevant problem structuring methods, such as "problem / solution" or "pro / contra". In this way the individual knowledge can be given an internal structure.

___________________________________________________________________________________________

Example :

Answer:

"All administrative procedures should be possible digitally: identification via webcam, payment via mobile phone or online, eg a) confirmations such as life certificates, etc., b.) Driver's license and passport; c.) Dealing with the district court (land register, etc.) It would also be desirable to have a chat facility for questions such as is already offered by banks today. - Invoices such as those for kindergarten via email instead of paper. - Module changes for the kindergarten. - School registrations and requests to change areas in digital form. - Confirmations of residence. There are countless possibilities here to make the system more efficient, to conserve resources and ultimately to make processes faster.

Core statements isolated from this:

• Dealing with authorities should be made possible digitally through identification via webcam, payment via mobile phone or online
• Life certificate, driver's license and passport, errands at the district court (land register, etc.), confirmations of residence, school registrations, requests to change areas, module changes for the kindergarten

Assigned To: Finding 1:

DIGITAL OFFICIAL ROUTES OF THE CITY F.

In order to avoid unnecessary official visits, applications, registrations, etc. should be able to be processed online in the future. Specific mentions were: school registrations, change of districts, orders for garbage bags [...]

and

• A chat facility for questions like the one offered by banks today would also be desirable

Assigned to finding 2:

COMMUNICATION INITIATIVES AND AVAILABILITY

One of the concerns mentioned many times was the establishment of easier communication channels with the municipal institutions: the suggestion of a central app for appointment reservations was particularly popular. A live chat should complement the offer [...].

___________________________________________________________________________________

Results report

A results report (see examples) summarizes both the qualitative findings and the results of the quantitative queries. The report will be published before thePublication coordinated with the participants.

Tips

• In both the kickoff event and the mini-workshops, it is often a good idea to involve the participants in small games. A selection of games, some of which are suitable for the purposes of a Deliberative Delphis Process, is available from gamestorming.com
• Mind mapping is a solution for creating and clustering key messages, which is particularly suitable for workshops. If you use a mind mapping software in the "Concept Mapping" mode, the core statements can be represented by individual nodes, while the associated original contributions are created as (hidden) notes on these nodes. In asynchronous settings, it is advisable to use tools that have been developed for the purpose of so-called qualitative content analysis and that have features to simulate the process of creating core statements.
• Problem structuring methods can be used both to find a good question and to structure findings.
• There are so-called Kanbans for many thematic occasions. A selection of these can be found in the following toolkid from the UK innovation agency Nesta: http://gonano-project.eu/wp-content/uploads/2018/05/DIY-Toolkit-Full-Download-A4-Size.pdf
• The moderation manual by Wagner, H., Seifert, JW, & Flockenhaus, U. (2009) offers a good introduction to problem structuring methods. Visualize - present - moderate: The classic - 29th edition (23rd, unchanged edition). GABAL and the book chapter: Garrette, B., Phelps, C., & Sibony, O. (2018). "Structure the Problem: Analytical Frameworks". In: Cracked it! (Pp. 95-116). Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-319-89375-4_6
• The booklet offers a particularly successful compilation of methods of visualized problem structuring in the form of diagrams: Eppler, MJ, Kernbach, S., & Pfister, RA (2016). Dynagrams- thinking in stereo: think sharper, work together more efficiently and communicate more clearly with dynamic diagrams . Schäffer Poeschel.
• Instructions for problem structuring with diagrams that are particularly suitable for reconstructing differences of opinion can be found in the journal article: Grötker, R. (2021). "Argument Mapping & Co. - Visualized problem structuring for dealing with dissent and uncertainty". Organizational Development, 40 (2), 46–54.

Why a Deliberative Delphi? Sense and purpose

The Deliberative Delphi is an instrument to benefit from the so-called “wisdom of the many”. In contrast to the classic approach of the wisdom of the many, which is based on the evaluation of quantitative information (namely the averaging), qualitative information is processed in a deliberative Delphi.

Concrete results of a Deliberative Delphi can consist of the following elements:

• Survey of deliberative attitudes
• Collection and analysis (coding / clustering) of arguments put forward
• Prioritization of fields of action by participants
• Multivariate analysis of response sequences: Are there recognizable patterns in how participants answer a sequence of questions?
• Plausibility check of arguments
• Evaluation of prima facie arguments vs. all-in-all-considered evaluations; Development of criteria for weighing up competing decision-making principles
• Analysis of extremes in consensus or dissent when evaluating arguments as an indicator of uncertainty in factual judgment
• Plausibility assessment for future scenarios or estimation questions (keyword "wisdom of the many")
• Co-creation of products and processes

Examples

Bürgerdelphi germ line therapy: pattern in Anword sequences

At the Bürgerdelphi germline therapy (Buedeka), participants formed an opinion on the legal regulation of germline therapy. Germ line therapy (which is carried out on the unborn embryo) is characterized by the fact that it has effects for later generations. These can be desired effects (such as the repair of hereditary diseases) or unforeseen undesirable effects. One question related to the regulation of basic research in Germany. The question was: Should it be possible for scientists in Germany to take part in basic research on germ line therapy?

15 out of 20 participants (75 percent) were in favor of enabling basic research on germ line therapy in Germany as well. The reasons given for the decision are interesting. This shows an interesting pattern in the response sequences. It can be seen that participants who differ on other points of opinion nevertheless come together in their opinion on the regulation of basic research - which indicates an interesting scope for possible compromises:

“Research will advance, if it is banned in Germany, then in other countries. And it would be a great shame if neither as a scientist nor as a legislator could have any influence on it. It is better to allow basic research to be controlled and subject to certain conditions than that it takes place somewhere in secret. "

“Germ line therapy is an opportunity to cure the most serious hereditary diseases such as cystic fibrosis. The known risks can be controlled sufficiently well within the framework of the existing legal system. A ban on basic research would be disproportionate. "

KIRA: co-creation

In the KIRA project (a consultation of stakeholders and experts on artificial intelligence in radiology - using the example of diagnosis, image analysis and recommended therapy for cardiac symptoms), participants in co-creation outlined solutions for previously identified problems. One set of solutions concerned the informed decision problem. An informed decision in a medical context means that patient values are included in the choice of diagnostic procedures and therapies. As part of the Citizens Delphi, three different approaches for the informed development were proposed and worked out in sketch form:

Variant A: The AI system supports the doctor in choosing a diagnostic procedure, in which the AI system provides objective probability figures for the risks of the diagnostic procedure, for the further development of the course of the disease and treatment opportunities. Doctors can share these numbers with patients.

Variant B: The AI system supports medical staff in communicating with patients so that decisions can be made based on their personal preferences (which are previously recorded).

Variant C: The AI system directly supports patients in making independent decisions.

Citizen Delphi Health Data: Balancing Competing Decision-Making Principles

At the Citizens Delphi Health Data, participants developed a position on the question of whether health data stored on the electronic patient record (EPR) should be freely available in a hypothetical scenario of health research - in the form that patients who do not want this should be active have to contradict. Such a solution is known as an “opt-out” model.

In general, the participants are open to the possibility of providing their health data from the ePA to research without exception. A large part of the participants also advocated use by research-based pharmaceutical companies. Nevertheless, the majority of the participants spoke out against the legal introduction of an opt-out solution. What was particularly striking was the observation that the participants spoke out against an objection solution and against the abolition of earmarking even if they expected medical data to be used in scientific and medical researchspeed up the step, and if they assessed that a change from an approval to an objection solution (i.e. from opt-in to opt-out) would significantly improve data availability. Reasons for this attitude are presented in detail by the participants in an argumentative manner. They underline the need for a solution that balances the competing values of individual freedom and social benefit.

Appendix:

Method comparison: what is the difference between the Deliberative Delphi and a Citizens' Council?

In contrast to the Citizens' Council, in which a group of participants (usually determined by lot) come together in plenary for a time slot of a weekend or several day sessions, the participants in the Deliberative Delphi work together in alternating members in mini-workshops that have one Be carried out for a period of several months.

In addition, the two methods differ in terms of how they work together. While citizens' council procedures use the mode of joint action (collaboration), the deliberative Delphi predominantly takes place in the mode of participation (contribution). In collaboration, the results of team members and participants are largely developed independently; the moderator only takes responsibility for process design and time management. With the contribution, the degree of participation is lower. In the deliberative Delphi, participants decide (in contrast to a pure survey) about the setting of the topic and about final decisions and positions. The analysis of the participants' contributions and the editing of the results report is carried out by the moderation team. The incentive economy is different in cooperation in the mode of participation than in the mode of joint action. Working together is based on mutual achievement. As soon as reciprocity is interrupted, teamwork comes to a standstill. In the mode of participation, on the other hand, the team members switch to the role of advisors and supporters who voluntarily provide their expertise.

The results also differed accordingly. While the participants in a Citizens' Delphi usually reach a decision by consensus, the Deliberative Delphi goes deeper into the content. As a result, it is not the formation of opinions within the group that counts here, but the gathering of arguments and the detailed elaboration of ideas (see above, Why a Deliberative Delphi?).