Adventures in Multimodality (AIM): Narrating and arguing by images, words & sounds

Coordinator

dr. Charles Forceville

Members

ACLC staff: dr. Assimakis Tseronis 

Description

The role of static and moving images in presenting information and arguments has become more prominent in recent decades. This tendency straddles print and digital media, and challenges the primacy of the verbally conveyed message in public space. Increasingly, the sonic modality also helps frame, or spin, information, particularly through music. Assuming that all discourses are goal-driven, the group seeks to chart and analyze how multimodal discourses are structured and how they achieve their rhetorical and/or aesthetic goals. Its members draw insights from cognitive and pragmatic approaches to metaphor theory, argumentation and visual communication, and share a commitment to develop hypotheses, where possible systematically test these, and thus uncover pertinent patterns. More information can be found on the AIM blog http://muldisc.wordpress.com/ and via the members' Google Scholar Profiles.

Current research projects

Relevance Theory as the best model for analysing visual and multimodal mass-communication

Sperber and Wilson’s Relevance Theory (Sperber and Wilson 1995, Wilson and Sperber 2012, Carston 2002, Clark 2013), which has developed from the Gricean framework, holds that all communication comes with the presumption of optimal relevance to its addressee. It claims to work for all communication, irrespective of medium or modality. However, it has hitherto focused almost exclusively on the modality of spoken language, more specifically on face-to-face communication between two individuals. To make good on its promises, RT requires adaptation and refinement; it can become a better model for visual and multimodal communication than currently available ones in semiotics. After having written a number of papers and chapters on dimensions of such adaptations (Forceville 1996, 2005, 2009, 2014, Forceville and Clark 2014), Forceville is now working on a monograph on this topic. For more information, contact c.j.forceville@uva.nl

Rhetorical analysis and multimodal argumentation

Within the field of visual rhetoric (Hill and Helmers 2004), a number of studies have had recourse to categories from classical rhetoric in order to explain the persuasiveness of visuals with reference to the audience addressed and the cultural, social and political context. In the last twenty years, argumentation scholars have also paid serious attention to the role that visual images may play in argumentative discourse (Groarke 1996, 2002; Roque 2009; van den Hoven 2012). However, emphasis has hitherto been placed on the visual and verbal content and on the evidentiary or emotive role of images. A combination of insights from cognitive pragmatics, multimodal analysis (Bateman 2014), and argumentation studies (Van Eemeren 2010) is required to assess the role of non-verbal modes in argumentative communication from a rational, rather than a purely aesthetic and emotional perspective. Tseronis and Forceville are currently co-writing papers and preparing an edited volume, Visual Argumentation for Benjamins. Their co-edited volume Multimodal Argumentation and Rhetoric in Media Genres is scheduled to appear with Benjamins in 2017 or early 2018. For more information, contact a.tseronis@uva.nl

Multimodality and metaphor

Lakoff and Johnson (1980) claim we think metaphorically. But their cognitivist paradigm has paid little attention to non-verbal manifestations. In this ongoing project (e.g., Forceville 2005, 2006, 2011, 2014, 2016a, Forceville and Urios-Aparisi 2009, Bounegru and Forceville 2011, Forceville and Renckens 2013, Kromhout and Forceville 2013, Koetsier and Forceville 2014) Forceville publishes about dimensions of visual and multimodal varieties of creative and conceptual metaphor as well as other tropes. For more information, contact c.j.forceville@uva.nl

Multimodality and comics, cartoons & animation films

These media are almost completely “man-made,” and in many cases do not, or minimally draw on language. They are therefore ideally suited for studying how coded non-mimetic signs, such as emotion lines and text balloons in comics and manner of movement in animation film, convey significant information. This is an ongoing project (Forceville 2005, 2005, 2011, 2013, 2016,  Forceville, Veale and Feyaerts 2010, Forceville and Jeulink 2011, Forceville, El Refaie, and Meesters 2014). For more information, contact c.j.forceville@uva.nl

Published by  ACLC

29 March 2017