Problems and Purpose

Participants were asked to deliberate in response to the question “Under what circumstances, if any, should artificial intelligence be used in Australian health systems to detect or diagnose disease?” [1] 

Background History and Context

With the development and advancement of artificial intelligence (AI), it has found its way into many areas of life. This offers opportunities as well as concerns. In areas such as healthcare, there are potential opportunities for greater accuracy in diagnoses and treatment, but this needs to be considered within the parameters of how the technology should be governed. While AI in healthcare is already being used in Australia, it needs further consideration. [2] 

Organizing, Supporting, and Funding Entities

The Australian Centre for Health Engagement, Evidence and Values (ACHEEV) commissioned and ran the citizens’ jury. The Sortition Foundation recruited participants. 

There was also a project reference group providing advice and guidance in the lead up to deliberations, particularly when determining the remit.

Participant Recruitment and Selection

30 participants, recruited through random stratified sampling, took part in the citizens’ jury. Participants received a stipend for taking part and had their travel, accommodation and meals for the in-person meeting provided by the organisers. [3] 

Some groups of people were excluded from participation if they worked with AI, were in clinical occupations or did not speak English. [4]

Methods and Tools Used

Citizens’ jury 

What Went On: Process, Interaction, and Participation

The process was 18 days in total; 15 online and three in-person in Sydney. The organisers also provided support and training for participants to be able to take part meaningfully. This included providing participants with technology they needed to participate online. [5] It also included training and resources in critical thinking and responding to cognitive bias to better equip participants to take part in the deliberative process. [6]

Participants received information through a handbook, online documents, and four expert presentation recordings. Participants were also given the opportunity to ask questions of the experts on the first online day of the in-person deliberations and throughout the online deliberations. [7] 

The process took over three weeks, a longer time period than typical citizens' juries which take place over a few days. The first week began with introductions to the process and critical thinking and then proceeded to information presentations. It ended with small group discussions that included developing questions to ask the experts. The small groups were frequently mixed during the 18 days to ensure the distribution of ideas and perspectives.

The second week involved asking experts the questions posed by participants and further information sessions. This week also ended with small group discussions and included identifying any areas that needed more information.

The third week had experts respond to any further questions participants had and with input from external experts as needed. This was also the week where participants met in-person from Friday to Sunday, culminating in approximately 18 hours' total meeting. As with the online discussions, the face-to-face deliberations began with introductions and expectations for deliberations, information presentations and then moved on to drafting and finalising recommendations. These recommendations were presented on the final day. [8]

Deliberations also included observers from external agencies that could help with "impact" from the initiative. [9]

Influence, Outcomes, and Effects

Participants distilled their deliberations down to 10 issues and 15 recommendations. The main priority recommendation was an “overarching, independently governing charter and framework” for using AI in healthcare. [10] 

Analysis and Lessons Learned

A formal evaluation of the process is being undertaken, but the organisers, overall pleased with the process, have published some reflections in the interim [11]

• They found that to make the citizens' jury as inclusive as possible required a lot of time, careful planning and resources to accommodate everyone
• Yet, this inclusion led to the great benefit that was a result of having such a diverse group of participants
• Jurors had too much time after expert presentations to draft questions in response, resulting in a large amount of questions and answers, and slowing the process down. Organisers noted that next time they will present expert presentation videos "live" and allocate a short time for responses
• While small group work and deliberations are helpful, particularly for larger juries, it does result in a disconnect between what facilitators are exposed to in deliberations. This does, however, mean greater agency for jurors to present to organisers what they deem is most important
• While this process was longer than many typical citizens' juries, facilitators still felt the process would have benefitted from more time, making specific mention of "wordsmithing"
• The hybrid approach of this jury and the increased small group activities meant all jurors engaged deeply with the information provided
• The hybrid approach also meant increased involvement at lesser cost
• Taking time to explain and discuss the process itself, as well as have a social event on the first in-person meeting, meant jurors were very collegial and supportive towards each other, as well as more open about their experiences of the process
• Organisers had to be responsive to the participants and changing dynamics of the deliberative process

See Also

References

[1] Carter, S. M., Aquino, Y. S. J., Carolan, L., Frost, E., Degeling, C., Rogers, W. A., Scott, I. A., Bell, K. J. L., Fabrianesi, B., & Magrabi, F., (2024) How should artificial intelligence be used in Australian health care? Recommendations from a citzens’ jury, Medical Journal of Australia, 220(8), 389-437, https://doi.org/10.5694/mja2.52283 

[2] Mayhew, K., (2024, May 28) AI will transform Australian healthcare. What should we do to get ready?, The Stand, University of Wollongong, https://www.uow.edu.au/the-stand/2024/ai-will-transform-australian-healthcare-what-should-we-do-to-get-ready.php 

[3] Carter, S. M., Aquino, Y. S. J., Carolan, L., Frost, E., Degeling, C., Rogers, W. A., Scott, I. A., Bell, K. J. L., Fabrianesi, B., & Magrabi, F., (2024) How should artificial intelligence be used in Australian health care? Recommendations from a citzens’ jury, Medical Journal of Australia, 220(8), 389-437, https://doi.org/10.5694/mja2.52283 

[4] Ibid. 

[5] Ibid. 

[6] Ibid. 

[7] Ibid.

[8] Ibid.

[9] Ibid.

[10] Ibid.

[11] Ibid. Supporting Information

External Links

Notes