Gestural Coding During Reading

enne_son's picture

In a paper published October 2012, Kimihiro Nakamura, Wen-Jui Kuo, Felipe Pegado, Laurent Cohen, Ovid J. L. Tzeng, and Stanislas Dehaene hypothesize “that, across all different cultures, the mature reading network comprises both a visual shape analysis system and a motor gesture decoding system.” They suggest that “this extended network is universal and that only its amount of activation is modulated according to the variable processing demands of writing systems. A motor memory for writing […] is known to play a general role in literacy acquisition in alphabetic literacy acquisition. Such gestural coding during reading may rely on a fast neural pathway […] that automatically recovers the intended motor gestures underlying visually perceived handwritten traces […].

The paper is titled “Universal brain systems for recognizing word shapes and handwriting gestures during reading,” See: www.pnas.org/cgi/doi/10.1073/pnas.1217749109

“[O]our results support the proposal that the expert reading network universally comprises two distinct pathways: an orthographic decoding system in the VWFA (reading by eye) and a kinesthetic gesture code system in Exner’s area (reading by hand). […] Our view predicts that the gesture code of writing should play a much greater role in reading when the expert VWFA system is not fully developed, either in early stages of reading acquisition or in the earlier ages of the literate human history when handwriting was used as a dominant medium for writing. Indeed, recent developmental data show that reading acquisition is facilitated when young children are taught to write or finger-trace the letter shapes compared with classical grapho-phonemic teaching without a haptic component. Conversely, fMRI of normal and dyslexic children also suggests that reading difficulties lead to a greater reliance on the left Exner’s area, suggesting partial compensation through the gesture system.

“Our results further suggest that a large part of the known cultural variations in the cerebral reading network may represent merely a differential weighting of the universal visual and gestural coding systems. Some writing systems, such as Chinese, Japanese, or Tamil, remain strongly centered on a realistic depiction of handwriting strokes, whereas others, including the Roman alphabet, are stylized to the point of using unique shapes for printed letters that even expert literates can no longer write (e.g., the shape of a printed letter g). We suggest that the former systems more systematically engage the gestural reading system […]. […] In the present study, this region, although contributing to both cultures, was more strongly engaged by moving than by static words in Chinese than in French participants, possibly because motor memory clearly plays a key role in memorizing the thousands of characters needed for fluent Chinese reading. Conversely, the VWFA, although universally involved in reading, is more activated in English than in Italian, presumably because of the greater number of graphemes needed in a nontransparent compared with a transparent alphabetic language.

[…]

[A]t a […] macroscopic level, we propose that cultural variability lies primarily in the different emphasis that distinct writing systems place on the visual and gestural pathways, thus resulting in modulations of the spatial extent and amplitude of brain activity within culturally universal brain circuits."

John Hudson's picture

Thanks for posting this, Peter. Very interesting.

It makes sense, of course, that over the long period in which, as Bringhurst notes, everyone who was a reader was also a writer, the reading process would have harnessed a motor gesture decoding system. A lot of interesting questions arise from this, especially with regard to the variable balance of visual and gestural pathways in the context changing text cultures. Are we, for instance, in the midst of a change in that balance due to the relative infrequency of handwriting and its secondary role in text creation in our culture?

J Weltin's picture

Thanks Peter.

One question in regard to this statement: Indeed, recent developmental data show that reading acquisition is facilitated when young children are taught to write or finger-trace the letter shapes compared with classical grapho-phonemic teaching without a haptic component.

Would the haptic component also be enhanced by letting children learn to write (i’m speaking of Latin languages/scripts now) their script with a broad-nibbed pen to better distinguish the pattern of letters? Nowadays they have to write their letters (very close to printed letters with almost no handwritten origin) with ballpoint pens showing no stroke contrasts, resulting in writing and reading difficulties.
Or am i moving too far from the above thesis?

Nick Shinn's picture

At last, scientific proof that English is the most advanced language!
This ties in with the “harder to read makes you smarter” research.

But seriously, such biological measurement of cultural differences makes me uneasy, like phrenology.

hrant's picture

{To Follow}

quadibloc's picture

@Nick Shinn:
At last, scientific proof that English is the most advanced language!
This ties in with the “harder to read makes you smarter” research.
You fools! This is why the Chinese will leave you behind in the dust!
Ah, but Chinese characters can still be written with brushstrokes by real human beings, whereas Latin letters can only be passively viewed because of their typographic stylization - thus depriving us of an aid to learning.
One doubts that makes up for the difference between English spelling, bad as it is, and the complexities of a character-based writing system.

Chris Dean's picture

I’m not sure I follow “Latin letters can only be passively viewed…” What was I doing when I made my grocery list?

quadibloc's picture

@Chris Dean:
What was I doing when I made my grocery list?

You're quite right, but the passage quoted did make that claim in essence: because Caslon or Times Roman is established as the normative form of the Latin alphabet, people don't get the assistance in learning it from the experience of forming the letters when writing them.

So, I suppose this means that Futura is more readable than Times Roman.

Chris Dean's picture

I’m still not sure I understand your position. Are you saying that in the contexts of the Latin alphabet frequency of exposure to Times/Caslon nullifies the brains ability to form a connection between the processes of reading and writing because the average person’s handwriting doesn’t have serifs?

If that were true, could I not point to the subtle differences between a Chinese character rendered by brush strokes compared to something mechanical, and make the same argument?

quadibloc's picture

This wasn't my position, I was merely echoing what was in the original excerpt:

Some writing systems, such as Chinese, Japanese, or Tamil, remain strongly centered on a realistic depiction of handwriting strokes, whereas others, including the Roman alphabet, are stylized to the point of using unique shapes for printed letters that even expert literates can no longer write (e.g., the shape of a printed letter g).

So, if this were true, our script would be harder to learn, because we can't handprint the kinds of letters we're used to reading, so the haptic component in learning to read is weakened.

Actually, though, reading more of the excerpt, the haptic component is mostly important as a substitute for the dyslexic, not as a primary requisite.

oldnick's picture

we propose that cultural variability lies primarily in the different emphasis that distinct writing systems place on the visual and gestural pathways

This is, perhaps, the single most preposterous claim ever posted in this forum…or the most brilliant. I am of two minds on the subject…

dezcom's picture

Gestural does not require thick-and-thin drawing. A line can have a gesture and be of one weight. Matisse has demonstrated this in some of his work.
Most handwriting in modern times is written with a tool that is not broadnibbed. True, weight of pressure and speed of stroke can make a line fade to some degree, even with a rapidograph.
If a child learning to write, traces the pathways of letterforms, it makes sense that this can take part in the imprinting of the form as well as the visual learning. I have a hard time seeing how this would vary with culture--Even if a given culture is more prone to using a brush instead of a ballpoint pen. It would seem that whatever makes a cultural variance is a function of the typical tool mixed with the script.

Nick Shinn's picture

… recognizing word shapes …

This from the paper’s title.
Are the researchers not familiar with the state of the art in reading science?

Cognitive psychologist Kevin Larson, Reading Researcher at Microsoft Corporation, has concluded that:
“Word shape is no longer a viable model of word recognition. The bulk of scientific evidence says that we recognize a word’s component letters.”

—Kevin Larson, The Science of Word Recognition, Eye magazine, Summer 2004

John Hudson's picture

Nick, I don't think they mean word shape in the sense of a whole word unitised recognition model. Their text refers to 'orthographic decoding system', which suggests something squarely within the letter-to-word recognition model.

Nick Shinn's picture

Right, but it’s nonetheless a faux pas to use the phrase “…recognizing word shapes…” in the title.

John Hudson's picture

Yes. I think they mean simply 'recognising words'.

oldnick's picture

Nick, Nick, Nick…

Your “component letters” theory is so…2004. Surely, in the ensuing nine years, the ideographic theory has regained some of its lustre (a tip o’ the Hatlo hat to the Brits in the forum), if not panache , élan and/or joie de vivre (likewise to the Frenchies).

In the end, how we read has nothing to do with what shapes culture; it’s what we read, and—increasingly—what we don’t read but are incessantly exposed to, that shapes culture…such as it is…

Syndicate content Syndicate content