[READING NOTES] Towards methods for evaluating and communicating participatory design: A multimodal approach

A journal article by Laura Malinverni, Joan Mora-Guiard, Narcis Pares

Summary and notes

The authors explore and implement methods to evaluate results of a participatory design process with children with special needs.

Problem: Participatory Design is highly demanding for children with Autistic disorders, and the results of a collaboration can be complicated to interpret. There's a lack of methodological tools for collecting and analizing stakeholder's contributions.

Solution : using a framework derived from multimodal analysis to collect and interpret user's contibutions during PD and conveying them to researchers. This method help guiding the discussion on interaction design, content definition, esthetics, consistency with educational requirements and design choices.


Case study:

Workshop to refine the design of a serious game based on FUll-Body interaction. How to use game mechanism in a non-behaviorist perspective? How to give space to children's contributions? Children are informants and give inputs/feedbacks in the design process.
They used "Wizard of Oz" as a technique to try in real time to adapt to children's reactions. Two activities were proposed: explore and draw (these are modal affordances to get a grasp on the prototype).

History and related work:

Participatory Design emerged in the 70es and has known an exponential growth over the last years. It is rooted "at a political and philosophical level, the roots of PD can be traced in postmodern tradition, phenomenology, Marxism and feminist epistemologies (Frauenberger et al., 2014 and Harrison et al., 2011)."

How to evaluate Participatory Design techniques? Frauenberger et al. (2014) define a "tool-to-think-with" composed of four aspects: epistemology, values, stakeholders, outcomes. Mazzone propose two dimensions: suitability and capabitlity. Moser and Schaper evaluates the content of the children's propositions.

Method: About multimodal analysis

It's derived from socio-semiotics and aims at analyzing situated interactions. It's grounded in the concept of mode, a set of socially and culturally shaped resources for making meaning. "Each mode has a set of modal affordances, which refers to “what is possible to express, represent or communicate easily with the resources of a mode and what is less straightforward or even impossible”"

"To analyze the data, one researcher first transcribed the videos according to a narrative style, in order to have an overall view of the experience. After that, she performed specific transcriptions, using different formats, for each unit of analysis (e.g. different forms of graphical transcriptions)."

Three concepts from multimodal analysis were used:


Children have similar exploration strategies, and concentrate on the nest. Design considerations: how to promote eye contact between peers? How to reward variability? Their interpretation of the prototype show that they identified goals that weren't designed by the researchers "(e.g. “You have to collect animals to make points”)." It "suggests that the presence of a narrative “mission” structure could facilitate children׳s understanding of the experience."

The use of multimodal analysis with participatory design has thus allowed to gather insights on design dimensions often left out. Furthermore, "combining the observation of embodied and situated meaning making with the analysis of children׳s productions, showed to be an effective approach to deepen on the meanings that children attribute to their actions and their interpretations of the experience." " We suggest that future research should address the analysis of the construction of specific socio-cultural values and the ethical reflection on the dilemma between reproducing them or diverging from them."

The interesting point of this research is that the outcomes of the design process were more easily shared within an interdisciplinary team. Recording techniques and material must thus be consistent with the context of the project.

Further reading