Grasping action in multimodal transformative processes

Fredrik Lindstrand, Konstfack University of Arts, Crafts and Design, Stockholm

A multimodal and social semiotic (Hodge & Kress, 1988) approach to learning, focusing on semiosis and semiotic change, provides an important contrast to the fields of research that currently inform political discourse on education and learning. By conceptualising learning as socially situated processes of sign-making and approaching learners as meaning-makers engaged in semiotic work (Kress, 2003; 2009), social semiotics affords valuable possibilities to grasp the social and epistemological complexities of learning and education (Insulander & Lindstrand, 2013; Insulander, Kjällander et al., 2017). In a world of instability and change, this seems as crucial as ever (cf. Kress, 2008).

However, approaching learning in ways that utilise the potentials of social semiotic theory calls for a research design that opens not only for analyses of signs and resources, but also for grasping sign-making as a process of decision making in situ over time (cf. van Leeuwen, 2005; Lindstrand, 2010). Differently put, it is a matter of balancing the two sides of social semiotics: the functional/social and the systemic parts of semiosis (Machin, 2016).

Building on examples from two research projects, the paper suggests that ethnographical approaches may offer ways to orchestrate this in practice (see also Dicks, Soyinka & Caffrey, 2006; Dicks, Flewitt et al., 2011). One of the projects, Making difference (Lindstrand, 2006; 2009) used ethnographic approaches to show how understandings of aspects related to ideational, interpersonal and textual features of communication with moving images were construed gradually in the transition between different phases, modes and media in collaborative filmmaking processes. The other project, The Mission (Lindstrand, 2016), used ethnographic approaches to track how various elements from a convergent learning process about WW2 were used as resources in the collaborative production of a written fictive story.

References

Dicks, B., Soyinka, B. & Coffey, A. (2006) Multimodal Ethnography. Qualitative Research 6(1), 77-96.

Dicks, B., Flewitt, R., Lancaster, L. & Pahl, K. (2011) Multimodality and ethnography: working at the intersection. Qualitative Research 11(3), 227-237.

Hodge, R. & Kress, G. (1988) Social semiotics. Ithaca, New York: Cornell University Press.

Insulander, E., Kjällander, S., Lindstrand, F. & Åkerfeldt, A. (eds.)(2017) Didaktik i omvandlingens tid. Text, representation, design. [Didactics in times of transformation. Text, representation, design]. Stockholm: Liber.

Insulander, E. & Lindstrand, F. (2013) “Towards a social and ethical view of semiosis. Examples from the museum”. In Böck, M. & Pachler, N. (red.) Multimodality and Social Semiosis: Communication, Meaning-making, and Learning in the Work of Gunther Kress. New York: Routledge. 225-236.

Kress G. (2003) Literacy in the New Media Age. London: Routledge.

Kress, G. (2008) Meaning and learning in a world of instability and multiplicity. Studies in Philosophy and Education 27(4), 253-266.

Kress, G. (2009) Multimodality. A social semiotic approach to contemporary communication. London: Routledge.

Lindstrand, F. (2006) Att göra skillnad. Representation, identitet och lärande i ungdomars arbete och berättande med film [Making difference. Representation, identity and learning in teenagers' work and communication with film]. Diss. Stockholm University. Stockholm: HLS Förlag.

Lindstrand, F. (2009) "Lärprocesser i den rörliga bildens gränsland" [Learning processes in the marches of filmmaking], in Lindstrand, F. & Selander, S. (eds.). Estetiska Lärprocesser – upplevelser, praktiker och kunskapsformer [Aesthetic Learning Processes - Experiences, Practices and Forms of Knowledge]. Lund: Studentlitteratur. 153-174.

Lindstrand, F. (2010) Interview with Theo van Leeuwen. Designs for Learning 3:1-2, 84-90.

Lindstrand, F. (2016) Med berättelsen och berättandet som mål och medel i en gränsöverskridande lärprocess kring andra världskriget. [Story and storytelling as target and means in a cross-boundry learning process about WW2]. Project report. Sandviken: Litteraturhuset Trampolin.

Machin, D. (2016) The need for a social and affordance-driven multimodal critical discourse studies. Discourse & Society 27:3, 322-334.

van Leeuwen, T. (2005) Introducing Social Semiotics. London: Routledge.