The Visual Language of Comics

Drawings and sequential images are an integral part of human expression dating back at least as far as cave paintings, and in contemporary society appear most prominently in comics. Just how is it that our brains understand this deeply rooted expressive system? I will present a provocative theory: that the structure and cognition of drawings and sequential images is similar to language. Building on contemporary theories from linguistics and cognitive psychology, I will argue that comics are “written in” a visual language of sequential images that combines with text. Like spoken and signed languages, visual narratives use a systematic visual vocabulary, strategies for combining these patterns into meaningful units, and a hierarchic grammar governing coherent sequential images. We will explore how these basic structures work, what cross-cultural research shows us about diverse visual languages of the world, and what the newest neuroscience research reveals about the overlap of how the brain comprehends language, music, and visual narratives. Altogether, this work opens up a new line of research within the linguistic and cognitive sciences, raising intriguing questions about the connections between language and the diversity of humans’ expressive behaviors in the mind and brain.

Microsoft Research

Source