IS COMIC COGNITION A GORDIAN KNOTTED ENIGMA?

These random streams of fractal thoughts are written down as a journal for reflection and learning in my experiential Unlearning and Relearning Journey. Then, if it provided any value to my readers or subscribers along the way, so be it! Please remember these are not Truths, and these newsletters are not opinions of an expert.

These fractal thoughts are inspired by Venu Gopal Nair post on comics: Comic Cognition is a real word in psychology.

I am not sure if my readers are aware that there is no consensus definition for “comics.”

"Comics are sometimes four-legged and sometimes two-legged and sometimes fly and sometimes don't?to employ a metaphor as mixed as the medium itself, defining comics entails cutting a Gordian-knotted enigma wrapped in a mystery?..." - RC Harvey 2001

Comics?is a?medium?used to express ideas with images, often combined with text or other visual information. It typically takes the form of a sequence of?panels?of images. Textual devices such as?speech balloons,?captions, and?onomatopoeia?can indicate dialogue, narration, sound effects, or other information.

Comics are NOT a language. Comics are written in visual languages in the same way novels or magazines are written in English.- Neil Cohn

Sequential image comprehension or “comic cognition” has received a good amount of focus in cognitive studies recently, and it is throwing up a wealth of questions to future researchers. They all focus on perception and attention(focus), and more importantly, the concept of event structure and multi-modality.

I am convinced our reality is made of panels of “comic events”?(?or is it “ cosmic events”?) weaved together in a unique event structure, like a movie is made of 18 panels per second, and editing organizes the event structure. Is quite possible that some humans have a poor reality editor inside their minds? is

I grew up in Chennai, gobbling up comics like Amar Chitra Katha, Irumbukai Mayavi, RK Narayan, Bobanum Mollyum, Asterix, and Tintin. I grew out of it and moved on to chapter books.

Being involved in a K-8 school in Michigan, I am aware of a case of a boy who was dyslexic when young, who was hooked on comics, and later, when he grew up to an adult, he could not move on to more complex books and that led to some complex cognitive issues.

Cyber “event structure” and the observables and modality are big topics in Cybersecurity Threat detection and response for me personally, also in Phishing Detection effectiveness NINJIO micro-learning security awareness videos, a brilliant initiative by Zack Schuler that uses comic storytelling structure very effectively, again a boring subject, I digress.

A few interesting questions below on comic cognition?

1. Are cartoony images processed differently than more realistic images?
2. “How is the form (i.e., graphics or sound) of the expressive system organized??
3. How is meaning conveyed by a form??
4. How do perceivers encode both form and meaning?”
5. How can simple changes in lines lead to drastic changes in meaning?
6. Are panels simply simulations of human attention? (Similar to attention focus)
7. If Yes, can research find cross-cultural differences in attention reflected in comics of different cultures (for example, Amar Chitra Katha in India)
8. How can visual representation of complex events inform our understanding of complex event structure and comprehension??
9. What can the tight and complicated interactions between the text and the image offer our understanding of multimodal interaction?

Quote Neil Kohn

Three primary components underlie the structure and definition of language, motivated by their cognitive architecture

1. “Modality—Languages are produced in a modality, be it creating sounds from the mouth (verbal), moving the body (manual/ bodily), or making marks on a surface (visual graphic). These expressions are, in turn, decoded by a sense organ (eyes, ears). While features of language may be (to some degree) transferable to other modalities (as in Braille or writing), natural languages have a predisposition for the modality they use.?
2. Meaning—Languages use modalities to express meanings. These meanings may be abstract or concrete and may use different manners of reference (discussed in the next chapter).?
3. Grammar—Languages use a system of rules and constraints for sequential expressions of meaning. If sequences obey these principles, we consider them to be comprehended acceptably (i.e., grammatically), while those that violate these principles read unacceptably (i.e., ungrammatical). In addition to these three primary components, language also has two important traits—it is both systematic and combinatorial. First, all of these components combine to create systematic units of expression. A “lexicon” or “vocabulary” of a language is built of systematic pieces of a modality (like sounds) that have meaning (as in words) and can be placed in rule-bound sequences (grammar). The most recognizable piece of a lexicon is a word, which is a systematic pattern stored in people’s heads. To the degree that these systematic pieces are shared between the minds of individuals, we say that they speak a common “language” in the world. Language is also a combinatorial system, allowing for these finite pieces to combine in infinite ways. For example, small units of the modality (sound) can be combined to create innumerable words, while units of the grammar (words) can be combined to create innumerable sentences. This combinatorial nature applies to all three of the traits of language: modality, meaning, and grammar.

Unquote

I know that Large Language Models do not have a cognitive architecture, specifically modality and meaning. Why do I keep wondering if people who create LLMs have cognitive architecture? I apologize; I digress.

Current News as comics is not new. "The common man is the creation of author and cartoonist RK Laxman. The Common Man, created in the year 1951, represents the hopes, aspirations, troubles, and, perhaps, even foibles of the average Indian. Laxman once said of his Common Man, “He’s been with me throughout my career. I did not find him. He found me… I would say he symbolizes the mute millions of India, or perhaps the whole world, a silent spectator of marching time.”

Is Human Cognition a Gordian Knot? Is comics a good knife to cut it?

Disclaimer: Opinions expressed in this article are solely my own and do not express the views or opinions of my employer.

Reference: The Visual Language of Comics: Introduction to the Structure and Cognition of Sequential Images. (Bloomsbury Advances in Semiotics) Book by Neil Cohn.