Multimodality and Microsoft Exhibition

Ok, I haven't quite finished the chapter when I'd hoped. This page on multimodality kept asking for more thinking, and then more... I finished last week, and was hopeful to finish the remaining two before posting all together, but because of travel (see below) I won't quite finish them till next week, so posting this solo now. This page follows directly from the page on visual-verbal interaction and brings another dimension to it in addressing the other modes, signs, symbols that comics incorporate to make meaning.

While they're not mentioned directly on the page, the content draws heavily on the work of Gunther Kress, Carey Jewitt, the New London Group, and more (all of this will be in my annotated appendix). The imagery took a lot of turns along the way - at some point i was drawing all the ingredients for an omelette (that's another story) - but finally by accident struck upon the confluence between a keyboard and orchestra. The hands arrived as I'd drawn an arrow to move the eye back up from the left sidebar of panels - but wanted something more organic and had previously been thinking of hands on the "keys." The thermometer, Descartes reference all go back to the opening to this chapter. Anyhow - I'll leave it at that, except to mention a few other things that prompted ideas. Paul Gravett's interview with John Miers (UK doc student) on his very cool Score and Script project, and this Russian comic/staff ("Etude No.81, " by I. Berkovich) - which I didn't actually draw on, but I really liked it and it served as inspiration so thought I'd share here. 

While I was working on this, the Graphixia, Comics and the Multimodal World conference was happening in Vancouver and while following along remotely, I was finding a lot of points of resonance with what I was working on.  Brooke Sheridan did a lovely write up of the gathering here. Also, Roger Whitson shared a preview of his presentation here. As always, this is a low-resolution version of my actual page - so some detail loss happens.

This week, I'm off to Seattle to give a talk and have an exhibition at Microsoft HQ in Redmond, Washington! The exhibition will feature a selection of printed works from the dissertation, as well as digital displays, and I'll be presenting on the work Friday, June 28. This is a new forum to share the work and I'm excited and grateful for the opportunity. Looking to share pictures and updates when I get back! Thanks for following along, till soon. - Nick

Visual-Verbal Intermingle

The slow but steady march to finish chapter three continues. Another new page, this time addressing how the unique ecosystem that is comics facilitates visual-verbal interaction - with mentions of the work of RC Harvey, David Lewis, and Edward Tufte. These and many other references I drew on to think through this page (including the excellent MLA session and subsequent special journal issue Charles Hatfield organized on comics and picture books) will all be listed in some sort of end note/commentary track in the completed dissertation. This page, as well as the last several and next three to come are all intended to demonstrate the thing they're talking about - not merely illustrate me talking about it. Thus, finding a form that achieves that has been an ongoing challenge to say the least! (I posted the entire sequence establishing my thinking and defining of comics immediately before this page here.) Even after I figured out the form on this one - tying back into the amphibious theme that opened the chapter, I still had to do things like draw "Imagetext" a few thousand times to make the water surface. (I did at some point take a short cut and copy a section of the text - though i think it added time because then i had to make all the lines coming to it align, which meant a lot of redrawing. Probably broke even in the end.) The reference to refraction also hearkens back to the opening sequence of the chapter and Descartes. I'm building on and re-weaving ideas together introduced earlier as I go forward, so while these can be read as stand alone pages, the payoff should be in the accumulation of all that's come before. (You can also compare this page to how I addressed similar ideas back in the journal article "The Shape of Our Thoughts.)

As always, this is a much lower res version than what I work at, and as a result some of the tiny detail in this page may not be visible. Thanks for following along - next post will be the finale of chapter three. Really. - Nick

A unique thought-space

Thrilled to finally be able to share this page, which represents the culmination of my thinking on this medium and along with the sequence leading up to it, really gets at the heart of my theorizing on why comics are cool! For some time, I've been working on comics as integrating the sequential and simultaneous reading modes, but it was through Lynda Barry's blog for her class on creativity that i came across Iain McGilchrist's work on the brain. In short, McGilchrist dispenses with the old left/right verbal/visual distinction and talks instead about them as fundamentally different ways of perceiving - which more or less correspond to sequential and simultaneous (you can learn more about his book "The Master and the Emissary here). Pieces just started to synch together. The background brain image refers back to the hierarchical tree/rhizome distinction from earlier in the chapter, which also aligns with this way of thinking about the hemispheres. Foreground left panels are redrawings of many of my sketches that led to this page - so you get a bit of behind the scenes at my thinking even as we're talking thinking... 

To get the whole sense of how I've been building this, I've included the pages immediately proceeding here, including my take on comics as a sequential art and their ability to convey the simultaneous. What follows from here to close out Chapter Three are the implications of this sequential-simultaneous fabric - how visual-verbal interaction works, multimodality, and how the space of the page itself is integral to the meaning within. Coming soon! Thanks for following along. - Nick