An efficient, reliable, and flexible technique for rating ideas, with instant results. This simple analog tool uses secret score-voting to recognize levels of agreement and avoid traditional voting problems such as the bandwagon effect, choice overload and vote-splitting.
Problems and Purpose
To quickly recognize collective opinions on many statements using a reliable and fun process, while avoiding groupthink and without requiring any technology. Works efficiently for large groups and dozens of ideas to find the ‘wisdom of the crowd’ with immediate transparent results, without relying on mobile devices.
Origins and Development
Feedback Frames were invented in 2014 by Jason Diceman, a then Senior Public Consultation Coordinate for the City of Toronto. He created the tool as a solution to the problems he saw in sticker dot-voting (dotmocracy). After years of prototype use within his own professional work, he began manufacturing and distribution of Feedback Frames in 2018. They are now in use by government bodies and by professional facilitators around the world.
The approach is based on score voting, and the likert scale, which have been proven effective in social science research since the 1950s. Also based on the “Gradients of Agreement Scale” developed in 1987 by Sam Kaner et al.
In 2004, Jason created an earlier version of the approach using pen to fill in dots on a scale of agreement in a paper form, originally called “Dotmocracy Sheets” and later “Idea Rating Sheets”.
How it Works
Feedback Frames are a secret score-voting ballot tool where participants vote by dropping tokens in a range of slots that are hidden by a cover, with results later revealed as a visual graph of opinions. See photos and video attached and at FeedbackFrames.com
Feedback Frames can be used at any stage of a participatory session, when there is a need to prioritize statements, such as:
- Questions to be answered
- Topics to be explored
- Proposals to be recommended
- Concerns to be addressed
- Suggestions to be considered
Rating scales and other materials can be easily customized for the specific language, topic and goals of the meeting. (See the library of free templates at FeedbackFrames.com/templates)
For example the standard rating scale provided is:
- Strong Agreement
- Agreement
- Neutral
- Disagreement
- Strong Disagreement
- ? - Not Sure
This prioritization technique can be used among a list of predetermined options, or following an participant idea generation process (e.g. brainstorm).
The technique scales up to allow for the rating of dozens of ideas at a time. While each frame only holds about 30 votes, it can still be used within a meeting of 100+ participants, as long as each frame receives a fair sample of voters. Ideas can also be duplicated across multiple frames for a larger sample.
The process requires between 30 minutes to an hour, including instructions, voting & commenting, and a review of the results.
Results may be used immediately in the next step of a meeting agenda, and/or recorded for sharing and consideration after the event.
Step-by-Step Instructions
(See photos above. For more detailed facilitation instructions, see FeedbackFrames.com/resources)
1. Ideas written on statement-signature sheets
Predetermined options and/or participant generated suggestions are written on statement-signature sheets (see template at FeedbackFrames.com/templates). Each sheet contains only one idea each.
2. Place sheets in front of Feedback Frames
Lay out the statement-signature sheets with statements on them in front of Feedback Frames set-out on tables.
3. Invite token dropping to rate ideas
For each idea, participants will silently read a statement and privately record their rating by dropping one single token in the corresponding frame along the scale provided.
They should also sign their name or initials on the right of each statement-signature sheet they vote on. Signatures track progress and will validate results by matching the number of tokens dropped per frame.
Optionally, they can also write comments on each sheet under the headings “Strengths & Opportunities”, or “Concerns & Weaknesses”.
4. Reveal the results
Once voting is sufficiently complete, it’s then time for the big reveal to “see what we think!” Reveals can generally be done all at once for efficiency, or one at a time for more suspense and rousing group reactions.
5. Record results as photos and token counts
Outside of the meeting agenda, a facilitator-assistant will photograph each individual frame result with its statement-signature sheet.
They will also write the token count for each column in pen at the bottom of each sheet.
6. Enter results data into a spreadsheet (optional)
If a session includes more than a dozen results, or if there is need to compile and compare results from multiple sessions, one can type up the statements, comments and token results into a digital spreadsheet. A simple formula can score each idea and then automatically sort by level of agreement. Add columns for themes and topics to help with analysis. (Download free spreadsheet templates at FeedbackFrames.com/templates)
Analysis and Lessons Learned
(pending)
External Links
See case studies, examples, templates, and resources at FeedbackFrames.com
Notes
The Feedback Frames design and method are patented: US20180189804A1
Feedback Frames are available for ordering internationally, with steep discounts for community groups, not-for-profits and public education institutions at FeedbackFrames.com