In multimodal theorizing, materiality refers to the fact that modes are taken to be the product of the work of social agents shaping material, physical ‘stuff’ into meaningful stuff, that is, into cultural / semiotic resources. This materiality has important semiotic potentials in itself: sound has different affordances to ‘graphic’ inscription; gesture offers different potentials to colour; and so on.
An underlying assumption in (most) approaches to multimodality is that all modes available in a culture are used to make meaning; and that modes are selected in designed ensembles to make the meanings that should best suit specific needs. All modes, on the basis both of their materiality and of the work that societies have done with that material – with sound becoming speech, or music; with movements of hands/arms displayed against the upper torso becoming gestures – offer specific potentials for making meaning and bring with them constraints. These affordances mean that all modes are partial in making meaning, so that the designed selection of the affordances of modes allows this partiality to be overcome. At the same time, the materiality of modes connects with the body and its senses. And while sensoriness is not a strong focus of most multimodal approaches, body and senses moves multimodality away, decisively from the intense abstractions of linguistic theorizing of the 20th century; and allows consideration of bodily response and physicality back into the domain of meaning.
In educational research, the notion of the differential affordances of modes has opened up the whole field of learning and assessment; of what knowledge is; how environments of learning should be designed; etc It makes it possible to attempt to recognize learning through means other than speech or writing. In media theories, in consideration of digital media with their use of a multiplicity of modes, in theories of interfaces, in design, etc the move away from an assumed dominance of writing leads to the possibility of new understandings.
Editor: Sara Price
Other contributors: Anders Björkvall and Gunther Kress
Björkvall, A. and Karlsson, A.( 2011)
‘The materiality of discourses and the semiotics of materials: A social perspective on the meaning potentials of written texts and furniture’
Semiotica, 187:1, 4, 141–165
(For a discourse view of materiality, artefacts and meaning making, and analysis of empirical data.)
Kress, G. and van Leeuwen, T. (2001)
Multimodal discourse: The modes and media of contemporary communication
(For discussions of materiality with regard to modes and media.)
van Leeuwen, T. (2011)
The language of colour: An introduction
London & New York: Routledge