In the interests of sparking commentary and kickstarting my nascent "design as cultural imperialism" thought process (and future unbook—because thoughts on social and cultural perception and construction are never complete), I'm excerpting portions of my Media Psychology and Social Change Master's capstone project on this blog. And yes, dear readers, together we have reached the series' conclusion. If you've arrived here by direct link, you can find the other posts here: Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7. As always, read, comment, tweet, retweet, argue, assert, agree: I'm game. And thanks for reading!

Conclusion
The next wave of media communication has arrived in the form of information visualization. Information visualizations are powerful ideological dissemination tools. By design they incorporate visual perception, contextualize data in parsimonious ways, inform cognitive and perceptual fluency, actuate cultural semiotics, and present data in explicit mental shorthand. In the new language of information visualization art and science coalesce to engender new understandings, and, if done really well, tell interesting stories. 

During the last seven posts, we've covered the complexities, rewards, and inherent dangers of data visualization. If, as we begin to explore this emerging language, we can simultaneously create and parse the visual neologisms that will arise, we will begin to alter what it means to talk about data, as well as what it means to present data. We may even alter our world in such a way that we will need to create verbal language to discuss this visual language.

The task that lies ahead offers huge opportunities, and as long as we can educate ourselves as to information visualization's potentials regarding perception, persuasion, and interpretation, we can use it without fear of its pitfalls. However, we will need to continually negotiate our way through the propagandistic fields of data, statistics, and graphic representation. The reward will be enrichment of not only human cognition but of human potential. For a new language provides new opportunities to think new thoughts, perceive new worlds, divine new connections, and exceed current mental boundaries.

Information visualizations comprise an intrinsically alluring area of study deserving far greater scientific attention and research from the psychological world than they currently enjoy. Fortunately, despite the current paucity of research, there does appear to be a surge in university programs focusing on the combination of media, communication, theory, design and programming.

In the coming years, the field of information visualization will prove to be a broad and intriguing field. And while psychological research into all aspects of data visualization seems inevitable, I hope these posts will further galvanize interest in studying it sooner rather than later.

 

 

In the interests of sparking commentary and kickstarting my nascent "design as cultural imperialism" thought process (and future unbook—because thoughts on social and cultural perception and construction are never complete), I'm excerpting portions of my Media Psychology and Social Change Master's capstone project on this blog. This is Part 7: potential issues and drawbacks. If you've arrived here by direct link, you can find the other posts here: Part 1, Part 2, Part 3, Part 4, Part 5, Part 6. As always, read, comment, tweet, retweet, argue, assert, agree: I'm game.

Some potential pitfalls of information visualization
As infographics surge in popularity, significantly divergent implementations are occurring in the wild: some of these alleged information graphics and data visualizations fail to pass the usefulness test. Indeed, some are mere eye candy that offer little to no cognitive benefit. In fact, they might be considered artistic or decorative graphics. These impotent efforts both weaken the impact and lessen the value of information visualization overall. The following paragraphs elucidate where and why information visualizations can go wrong. Any and all joking aside, given the significant cognitive power information visualizations can wield, they can be weaponized just as easily. If designed poorly or incorrectly, they can introduce a new spin on the old statistics saw, "lies, damn lies and information visualization."

Subjectivity, cultural bias and aesthetic appeal
For all of the practical, affirmative mental assistance visualizations provide, some potential dangers lurk within them. Data may be considered agnostic but interpretation of it is subjective and therefore biased by personal interest and cultural interpretation, regardless of intent. Consequently so are the visualizations that are created from them. While visualizations aim to contextualize information, they are, nevertheless, one step removed from the statistical data they seek to represent. This divergence, however small, can affect interpretation in any number of ways. For instance, the temptation to succumb to data mining—illustrating only the data that suits the hypothesis or conjecture (Tufte, 1997)—can be great.

Representing information graphically does not magically remove intervening variables, but rather alternatively reorients data in ways that may make intervening variables appear less important or less immediately troublesome. Conversely, visualizations may emphasize content that proffers somewhat less than essential or informative information. Social scientists and designers must beware the siren call of aesthetic inspiration that hinders the clear representation of data: statistical graphics and visualizations are tools  "to help people reason about quantitative information" (Tufte, 1983, p. 91). If a visualization complicates data or presents it in a weak or confusing manner, it increases cognitive load and decrements the ability of the representation to bolster the power of human perceptual processing (Norman, 1993).

 

Next up: Information visualization: a new visual language (part 8) Conclusions

 

References

Norman, D. (1993). Things that make us smart. Reading, MA: Addison Wesley.

Tufte, E. (1983). The visual display of quantitative information. Cheshire, CT: Graphics Press.

Tufte, E. (1997). Visual explanations. Cheshire, CT: Graphics Press.

 

 

Copyright © 2011 Carla Casilli. All rights reserved.

In the interests of sparking commentary and kickstarting my nascent "design as cultural imperialism" thought process (and future unbook—because thoughts on social and cultural perception and construction are never complete), I'm excerpting portions of my capstone project on this blog. You're reading Part 6: cognitive advantages of fluency. If you've arrived here by direct link, you might catch up here: Part 1, Part 2, Part 3, Part 4, Part 5. As always, read, comment, tweet, retweet, argue, assert, agree: I'm game.

Cognitive Advantages
What are the rewards for a bit of cognitive assistance? A mixture of internal and external knowledge affords us the luxury of reflection—modification, manipulation and comparison of mental representations or ideas (Norman, 1993). Visualizations can facilitate the scaffolding of large and complex issues, augmenting the contemplation of complex or previously hidden relationships, and correlating hitherto unrelated or disconnected data. "Representations are important because they allow us to work with events and things absent in space and time, or for that matter, events and things that never existed—imaginary objects and concepts" (Norman, 1993, p. 49-50). 

Specifically, diagrams offer computational advantages (Larkin & Simon, 1987) because the information contained within them is indexed in ways that "can support extremely useful and efficient computational processes" (p. 99). Taken a step further, "dynamic visualizations maybe considered as less complex than static visualizations since the depicted dynamics are readily perceivable and do not need to be inferred by the learner" (Schmidt-Weigand, 2009, p. 101). Visualizations can conceptually chunk information through visual grouping further speeding cognitive processing thereby reducing "the need for searching for multiple information elements related to a single idea" (p. 72). 

Properly designed visualizations present data in new forms that enhance "the ability to make judgments, to discover relevant regularities and structures" (Norman, 1993, p. 52).  This interpretive aspect is important because “it is through metarepresentations that we generate new knowledge, finding consistencies and patterns in the representations that could not readily be noticed in the world” (p. 51).

Since human brains have evolved to think using pictures, and the sequentiality of text may slow or impede cognition, it follows that "presenting information as pictures is the most efficient way to present information to people" (Weinschenk, 2009, p. 115). Ergo, good information visualizations—those that capture the essential elements while excluding extraneous ones—can transform problems into easy experiential tasks. This transformative capacity is valuable because "experiential artifacts thus mediate between the mind and the world" (Norman, 1993, p. 52). 

Ultimately, information visualization should encourage message processing through imaginal thinking. Paivio (1975) notes, "imaginal thinking can be characterized by remarkable speed, accuracy, and flexibility of information processing" (p. 161). Moreover, imaginal thinking quickly and synchronously organizes imagery into units that then function essentially as memory storage units (Paivio, 1975). "External visualizations enable cognitive operations that would otherwise have to be conducted internally (e.g., mental imagery)" (Scheiter, Weibe & Holsanova, 2009, p. 75).

Because they can compress huge volumes of data in real-time by providing deep, contextualized information at a glance, visualizations can dramatically reduce cognitive load and increase the human ability to absorb additional information: real wins for cognitive processing, encoding, and retrieval. 

Fluency, Familiarity and Visualization
Visualizations also traffic in the psychologically important currency of narrativity (as discussed in part 5) and fluency. Fluency, "the subjective experience of ease or difficulty with which we are able to process information" (Oppenheimer, 2008, p. 237), is "a ubiquitous metacognitive cue that accompanies cognition across the full spectrum of cognitive processes" (Alter & Oppenheimer, 2009, p. 232) affecting perception, conceptualization, linguistics, retrieval, encoding, embodiment, decision-making, spatial perception, deduction, generativity, and attention (Oppenheimer, 2008). 

The perception of fluency seems to elicit positive reactions toward the subject or experience perceived as fluent (Winkielman, Halberstadt, Fazendeiro & Catty, 2006). Hence, the potential fluency generated by the aesthetic appeal of visualizations may counterbalance some of the more formidable traits of large and complex data sets. Conversely, Reber, Schwarz & Winkielman (2004) theorize that fluency directly influences aesthetic perception. In other words, the more fluent an experience or object seems to a perceiver, the greater the perceiver's aesthetic response. Song & Schwartz (2010) contend that our sense of aesthetics is intertwined with an innate attraction to fluency. 

In general, research seems to indicate that fluency repeatedly trumps disfluency: highly fluent statements seem truer, more likable, more frequent, more famous, are better category members and seem to come from a more intelligent source than disfluent statements (Oppenheimer, 2008). Consequently, fluent visualizations are likely to possess those same positive qualities.

Somewhat unexpectedly, disfluency, while initially seeming like an undesirable trait, may be finessed to a communicator's advantage. Simple typographic disfluency in the form of a degraded font led some research participants toward a more "systematic processing strategy" (p. 239) and away from an automatic response. Consequently, the tendency of visual disfluency to affect or shift cognitive methodologies may play an important role in the design and display of controversial or cognitively complex data. 

And this brings us around to the potential importance of the mere exposure effect (Zajonc, 1968) in visualization. Although people generally prefer familiar stimuli to new but otherwise identical stimuli (Alter & Oppenheimer, 2009), through effective exploitation of the mere exposure effect they can be induced to accept disfluency and even like it if they are exposed to it frequently enough. The processing fluency/attribution model (Bornstein & D'Agostino, 1994) argues a similar point but includes fluency as part of the equation: the greater the frequency of exposure, the more easily the stimuli can be retrieved from memory which leads to positive attribution (Alter & Oppenheimer, 2009). Hence, an increase in the frequency of visualizations appearing in media sources could reflexively increase the audience's acceptance and appreciation for this type of data representation. 

 

Next up in Part 7: potential issues and drawbacks of information visualization. Part 8 will conclude our series. 

 

References

Alter, A., & Oppenheimer, D. (2009). Uniting the tribes of fluency to form a metacognitive nation. Personality and Social Psychology Review, 13(3), 219-235.

Bornstein, R. F., & D’Agostino, P. R.  (1994). The attribution and discounting of perceptual fluency: preliminary tests of a perceptual fluency/attributional model of the mere exposure effect. Social Cognition, 12, 103-128.

Larkin, J. & Simon, H. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65-99.

Norman, D. (1993). Things that make us smart. Reading, MA: Addison Wesley.

Oppenheimer, D. (2008). The secret life of fluency. Trends in Cognitive Sciences, 12(6), 237-241.

Paivio, A. (1975). Imagery and synchronic thinking. Canadian Psychological Review, 16(3), 147-163.

Reber, R., Schwarz,  N., & Winkielman, P. (2004). Processing fluency and aesthetic pleasure: is beauty in the  perceiver’s processing experience? Personality and Social Psychology Review, 8(4), 364-382.

Scheiter, K., Wiebe, E., & Holsanova, J. (2009). Theoretical and instructional aspects of learning with visualizations. In R. Zheng (Ed.) Cognitive Effects of Multimedia Learning (pp. 67-87). Hershey, PA: IGI Global.

Schmidt-Weigand, F. (2009). The influence of visual and temporal dynamics on split attention: evidences of eye tracking. In R. Zheng (Ed.) Cognitive Effects of Multimedia Learning (pp. 89-107). Hershey, PA: IGI Global.

Song, H., & Schwarz, N. (2010). If it's easy to read, it's easy to do, pretty, good, and true. The Psychologist, 23(2), 108-111.

Weinschenk, S. (2009). Neuro web design. Berkeley, CA: New Riders.

Winkielman, P., Halberstadt, J., Fazendeiro, T., & Catty, S. (2006). Prototypes are attractive because they are easy on the mind. Psychological Science, 17(7) 799-806.

Zajonc, R. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, Monograph Supplement, 9(2), 1-27.

 

 

Copyright © 2011 Carla Casilli. All rights reserved.

In the interests of sparking commentary and kickstarting my nascent "design as cultural imperialism" thought process and future unbook—because thoughts on social and cultural construction are never complete—I'm excerpting portions of my capstone project on this blog. You're reading Part 5: visualizations, semiotics and narrative. If you've arrived here by direct link, you might catch up here: Part 1, Part 2, Part 3, Part 4. As always, read, respond, argue, assert, agree: I'm game.

Visualizations and Semiotics
According to Scheiter, Wiebe & Holsanova (2009), visualizations can be sorted into three semiotic types:

• realistic—aim to physically resemble their referents;
• analogical—seek to represent phenomenal or abstract conceptual analogies;
• logical—describe charts, diagrams, and graphs; graphics whose spatial layout conveys information about conceptual relationships.

The same authors also assert that as text adjuncts, visualizations perform five functions:

• decorative—enlivening text
• representational—making information more concrete
• organizational—bringing coherence to data
• interpretative—enhancing understanding of data
• transformative—endeavoring to recode information into more concrete and memorable form, organizing information through relations, and enhancing retrieval of data by providing systematic means of retrieval.

Worth noting: despite their positive intentions, decorative visualizations can have counterproductive effects on processing and cognition. And of the five functions, the most complex, transformative visualization, shows "the strongest positive effect on learning outcomes" (p. 76).

The Narrative Potential of Visualizations
The depth of information embedded within a visualization permits it to act as both visual and conceptual shorthand, rendering it a sort of cognitive crib sheet. A good one does not necessarily veer into the world of make-believe, yet it can precipitate that process if that is its goal. Narrative functions in the same way, by bundling and transporting complex concepts in memorable ways, "Stories are important cognitive events, for they encapsulate, into one compact package, information, knowledge, context and emotion" (Norman, 1993, p. 129).

Effectively designed transformational visualizations can assume this exact role, acting not just as simple data representations but rather as layered, multidimensional, optical allegories, perceptually equivalent to stories. By tuning in the informational signal and tuning out the useless noise, visualizations create traversable bridges linking logic and narrative, economically presenting meaningful data.

Next up: Information visualization: a new visual language (part 6): cognitive advantages and the role of fluency.
I will address potential issues and drawbacks in Information visualization: a new visual language (part 7).

 

References

Norman, D. (1993). Things that make us smart. Reading, MA: Addison Wesley.

Scheiter, K., Wiebe, E., & Holsanova, J. (2009). Theoretical and instructional aspects of learning with visualizations. In R. Zheng (Ed.) Cognitive Effects of Multimedia Learning (pp. 67-87). Hershey, PA: IGI Global.

 

 

Copyright © 2011 Carla Casilli. All rights reserved.

In the interests of sparking commentary and kickstarting my nascent "design as cultural imperialism" thought process (and future unbook) I'm taking the opportunity to excerpt portions of my capstone project on this blog. You're reading Part 4. In this go-round we define what visualizations are, how they're cognitively efficient and why this might be important. Also included: more great references. Go here for Part 1. Go here for Part 2. Go here for Part 3. As always, read, respond, argue, assert, agree: I'm game.

Defining Information Visualization
Visualizations might be considered an extension of our natural proclivity toward notation, but what exactly are they? A very technical description might be, "A specific form of external representation...intended to communicate information by using a visuo-spatial layout of...information...processed in the visual sensory system" (Scheiter, Wiebe & Holsanova, 2009, p. 68). Visualizations provide a type of temporal integration—a simultaneous presentation of "multiple sources of [visual] information" (Ayres & Sweller, 2005, p. 143) that decrease extraneous cognitive load.

Barbara Tversky alleges that well-designed graphics, of which visualizations are a subset, pack a vast amount of information in a relatively small space:

degrees of relationship, such as similarity, salience, or strength, can be suggested spatially by degree of proximity, or pictorially by degree of appearance, color or size for examples. Proportion can be indicated by spatial proportion. Direction is conveniently conveyed by arrows, whether the direction is spatial, temporal, causal, or other. (Tversky, 2002, p. 19)

There’s a lot more to a good visualization than that, though. They're potentially tricky, as Donald Norman (1993) elucidates when he writes,

The critical trick is to get the abstractions right, to represent the important aspect and not the unimportant. This allows everyone to concentrate upon the essentials without distractions from irrelevancies. Herein lie both the power and the weakness of representations: Get the relevant aspects right, and the representation provides substantive power to enhance people's ability to reason and think; get them wrong, and the representation is misleading, causing people to ignore critical aspects of the event or perhaps form misguided conclusions. (p. 49)

The Economics of Good Design in Visualization
In The Visual Display of Quantitative Information, Edward Tufte (1983) declares, "Graphics reveal data" (p. 13). This elegantly succinct statement defines Tufte's philosophical ethos and serves as a universal starting point for information visualization. Tufte avers that properly designed statistical graphics make large data sets coherent through parsimonious presentation of data. By exposing multiple levels of detail in a visually simple manner, visualizations encourage data comparison and engender substantive thought. Economy seems to be an essential aspect to visualization. Donald Norman’s appropriateness principle can be applied to visualizations as well: "The representation used by the artifact should provide exactly the information acceptable to the task: neither more nor less" (Norman, 1993, p. 97).

Cognitive Processing: Effortful or Effortless
The way a problem is represented dramatically affects the perception of ease for that task, even though an alternative representation does not technically change the problem itself (Norman, 1993). One of the essential differences between verbal representation and visual representation is sequentiality. In 1987, Larkin and Simon contrasted sentential representations (words) with diagrammatic representations (images) and found that sentential representations are rigidly sequential while diagrammatic representations are indexed by location on a plane. The constraint of temporal sequential delivery affects both perception and cognition. Like physical separation, temporal separation—even moving from one word to another within a paragraph—can be supposed to generate extraneous cognitive load (Ayres & Sweller, 2005). The effort required to process visual material versus verbal information is markedly different: "reading requires word-by-word fixations, lexical access, and syntactic as well as semantic processing while the information depicted by visualizations may be gathered at a glance" (Schmidt-Weigand, 2009, p. 101).

Larkin and Simon concluded that whereas diagrammatic representations display information explicitly, sentential representations do so implicitly, "and therefore [information] has to be computed, sometimes at great cost, to make explicit for use" (Larkin & Simon, 1987, p. 65). They argue,

The major difference in a diagrammatic representation, we believe, is difference in recognition processes. We have seen that formally producing perceptual elements does most of the work of solving the geometry problem. But we have a mechanism—the eye and the diagram—that produces exactly these “perceptual” results with little effort. We believe the right assumption is that diagrams and the human visual system provide, at essentially zero cost, all of the inferences we have called “perceptual.” … It is exactly because a diagram “produces” all the elements “for free” that it is so useful. (Larkin & Simon, 1987, p. 92)

In contrast to language’s sequentiality, visualization can synchronously present multiple layers of information while contextualizing the information it presents. "[G]raphics can be more precise and revealing than conventional statistical computations" (Tufte, 1983, p. 13). From this standpoint, visualizations perform their functions in a parsimonious manner: "spatial relations of objects are automatically implied in a picture and thus do not require any additional symbols, whereas in verbal representation spatial arrangements need to be made explicit" (Scheiter, Weibe & Holsanova, 2009, p. 71).

 

Next up: Information visualization: a new visual language (part 5) The semiotics and narrativity of visualization.

 

References:

Ayres, P., & Sweller, J. (2005). The split-attention principle in multimedia learning. In R. F. Mayer (Ed.), Cambridge Handbook of Multimedia Learning (pp. 135-146). New York, NY: Cambridge University Press

Larkin, J. & Simon, H. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65-99.

Norman, D. (1993). Things that make us smart. Reading, MA: Addison Wesley.

Scheiter, K., Wiebe, E., & Holsanova, J. (2009). Theoretical and instructional aspects of learning with visualizations. In R. Zheng (Ed.) Cognitive Effects of Multimedia Learning (pp. 67-87). Hershey, PA: IGI Global.

Schmidt-Weigand, F. (2009). The influence of visual and temporal dynamics on split attention: evidences of eye tracking. In R. Zheng (Ed.) Cognitive Effects of Multimedia Learning (pp. 89-107). Hershey, PA: IGI Global.

Tufte, E. (1983). The visual display of quantitative information. Cheshire, CT: Graphics Press.

Tversky, B. (2002). Some ways that graphics communicate. In N. Allen (Ed.) Working With Words and Images: New Steps in an Old Dance. Westport, CT: Ablex Publishing.

 

 

Copyright © 2011 Carla Casilli. All rights reserved.

In the interests of sparking commentary and kickstarting my nascent "design as cultural imperialism" thought process (and future unbook) I'm taking the opportunity to excerpt portions of my capstone project on this blog. You're reading Part 3. This section reviews the problems inherent in human memory and the role visual representations play in it. This one is chock full of delicious references. I recommend investigating them on your own. Go here for Part 1. Go here for Part 2. As always, read, respond, argue, assert, agree: I'm game. 

The Trouble With Memory
As for memory, the visual sort wins hands down: the pictorial superiority effect documents our facility for image recall (Nelson, Reed & Walling, 1976). "We remember only ten percent of what we hear or read (without pictures)" (Weinschenk, 2009, p. 115). Moreover, memory is both selective and constructive, meaning that we construct our memories each time we retrieve them, basing our confabulations on original interpretations of meaning and completely disregarding the initial encoding methodology (Anderson, 2005). And while our memories begin their lives replete with particulars, they age poorly: the details that initially define them diminish rapidly. Consequently, despite our extraordinary image processing and retention capabilities, we have faulty memories. "Retrieval from long term memory is apt to be slow and to contain errors. Here is where information in the world is important, to remind us of what can be done and how to do it" (Norman, 1990, p. 191).

Dual Coding and Visuospatial Perception
Equally challenging to keeping a memory is obtaining one. Paivio (1970, 1975, 1991) broke new ground with his foundational dual coding theory that supposed two cognitive subsystems: one related to verbal processing and the other to non-verbal processing. According to his theory the two subsystems work in parallel to address stimuli. Several years after Paivio proposed his dual coding theory, Baddeley & Hitch (1974) hypothesized a three-component model of working memory (or short term memory): it theorized two storage systems, the visuospatial sketchpad and the phonological loop, and one control system, the central executive. In 2001, Baddeley (2003) rounded out his theory of working memory by adding a fourth component—the episodic buffer—the liaison between perception, long-term memory and action.

At first glance, Paivio’s and Baddeley’s theories might be considered competitors, but it’s perhaps more accurate to think of them as alternative interpretations of the same idea. Indeed, the commonality between the two is striking: both Paivio and Baddeley suggest that the mind uses different processing and storage techniques for verbal versus visual stimuli, although Paivio distinguishes between verbal and non-verbal.

Regardless of which cognitive theory you ascribe to, verbal stimuli appear to be differently processed than visual stimuli and seem to require more complex mental manipulation to encode than imagery does.

Groups of words apparently cannot be similarly integrated in memory [like images can]; instead, the units appear to be sequentially combined or concatenated into linear informational structures that take up more storage space and are subject to sequential constraints to a degree not characteristic of images. (Paivio, A., 1975)

"The human mind, for all its powers, is limited in its ability to think deeply about a topic, primarily because of the restricted capacity of working memory" (Norman, 1993, p. 246). Consequently, converting less meaningful material to more meaningful material greatly improves its memorability quotient. Likewise, "Memory for verbal material is greatly enhanced if one can develop visual images corresponding to the material" (Anderson, 2005, p. 108). To this end, representative visualizations can act as memory aids, greatly enhancing working memory capacity by decreasing or even eliminating extraneous cognitive load. External assistance such as information visualization achieves this by "representing an idea in some external medium…free from the limits of working memory" (Norman, 1993, p. 246).

It’s worth remembering, however, that the need for cognitively assisted mental reflection is both subjective and changeable. Low, Jin & Sweller (2009) note that experienced learners—those with some expertise in the area being learned—require fewer modes for learning. The modality effect they found suggests that information redundancy such as diagrams with explanatory text or text accompanied by explanatory diagrams may actually  impede cognitive processing. Intelligibility plays a role in processing, too, because "a representation is useful only if one has the productions that can use it" (Larkin & Simon, 1987, p. 70).

 

Next up: A new visual language (part 4): defining information visualization

 

References

Anderson, J. R. (2005). Cognitive psychology and its implications (6th ed.). New York, NY: Worth.

Baddeley, A. (2003). Working memory: looking back and looking forward. Nature Reviews Neuroscience, 4(10), 829-839.

Baddeley, A., & Hitch, G. (1974). Working memory. In G. H. Bower (Ed.), The Psychology of Learning and Motivation: Advances in Research and Theory Volume 8 (pp. 47-89). New York, NY: Academic Press.

Larkin, J. & Simon, H. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65-99.

Low, R., Jin, P., &  Sweller,  J. (2009). Cognitive architecture and instructional design in a multimedia context. In R. Zheng (Ed.) Cognitive Effects of Multimedia Learning, (pp 1-16). Hershey, PA: IGI Global.

Nelson, D. L., Reed, V. S., & Walling, J. R. (1976). Pictorial superiority effect. Journal of Experimental Psychology: Human Learning and Memory. Vol. 2(5), 523-528.

Norman, D. (1988/1990). The design of everyday things. New York, NY: Doubleday.

Norman, D. (1993). Things that make us smart. Reading, MA: Addison Wesley.

Paivio, A. (1970). On the functional significance of imagery. Psychological Bulletin, 73(6), 385-392.

Paivio, A. (1975). Imagery and synchronic thinking. Canadian Psychological Review, 16(3), 147-163.

Paivio, A. (1991). Dual coding theory: retrospect and current status. Canadian Journal of Psychology, 45(3), 255-287.

Weinschenk, S. (2009). Neuro web design. Berkeley, CA: New Riders.

 

 

Copyright © 2011 Carla Casilli. All rights reserved.

In the interests of sparking commentary and kickstarting my nascent "design as cultural imperialism" thought process (and future unbook) I'm taking the opportunity to excerpt portions of my capstone project on this blog. You're reading Part 2. This section reviews how vision, language and thought are twinned and intertwined. Go here for Part 1. As always, read, respond, argue, assert, agree: I'm game.

 The curious relationship between language and vision
Mayer & Massa's (2003) visualizer-verbalizer hypothesis states that some people process words more effectively (verbalizers) and some people process pictorial representations more effectively (visualizers). A closer evaluation reveals that visual-verbal inclinations influence cognitive styles as well as learning preferences. But the methods we use to acquire semantic and visual knowledge are nearly identical.

Just as children acquire rules of grammar without explicit teaching, so, too, do we acquire rules of vision (Hoffman, 1998). Our minds learn to construct the world we see, creating "visual worlds from ambiguous images in conformance to visual rules" (p. 24). Using these acquired rules, that range from interpolating stereoscopic vision to interpreting color, illumination and hue; from inferring distance to estimating motion; from cue integration to figure ground delineation, we learn to visually construct the world around us (Hoffman, 1998). The visual rules we learn are nothing short of astounding. These rules are occasionally misleading as several recent books and articles have pointed out but more often than not, they are absolutely essential to our ability to navigate and understand the world. The well-worn phrase seeing is believing is not far from the truth: what we think we perceive is how we learn to understand the world. Perhaps even more amazing is the extensibility of these visual rules since they "endow us with the capacity to understand countless images, even ones not yet seen by any person at any time" (p. 29). In other words, the rules permit us to parse unknown objects with relative cognitive ease.

Images as thought and communication
Visual rules allow us to perceive images, and coincidentally or not, are a vital component of our cognition. Antonio Damasio (1994/2005) asserts that "regardless of the sensory modality in which they are generated" (p.107), images are most likely the main content of thought whether or not the thoughts are about things, processes, words or symbols. Damasio does allow that words and abstract symbols might be mixed into image-based thought; however, he asserts that imagery remains the germinal cognitive building block because, "both words and arbitrary symbols are based on topographically organized representations and can become images" (p. 106).

Assumptions about image-based thinking notwithstanding, humans do seem innately predisposed to understanding external notational systems even if they are novel: "Cross-cultural studies as well as analyses of historical documents show communality in the way that elements and space are used in communication, suggesting that some graphic devices are cognitively natural" (Tversky, 2002, p. 4). Indeed, it appears that we have evolved to make marks and symbols to assist cognition, "People usually do this naturally: This is not some abstract, academic exercise" (Norman, 1993, p. 47).

References:

Damasio, A. (1994/2005). Descartes’ error: emotion, reason and the human brain. New York, NY: Penguin.

Hoffman, D. (1998). Visual intelligence: how we create what we see. New York, NY: W.W. Norton & Company, Inc.

Mayer, R., & Massa, L. (2003). Three facets of visual and verbal learners: cognitive ability, cognitive style, and learning preference. Journal of Educational Psychology, 95(4), 833-846.

Norman, D. (1993). Things that make us smart. Reading, MA: Addison Wesley.

Tversky, B. (2002). Some ways that graphics communicate. In N. Allen (Ed.) Working With Words and Images: New Steps in an Old Dance. Westport, CT: Ablex Publishing.

 

 

Copyright © 2011 Carla Casilli. All rights reserved.

In the interests of sparking commentary and kickstarting my nascent "design as cultural imperialism" thought process (and future unbook) I'm taking the opportunity to excerpt portions of my capstone project on this blog. Read, respond, argue, assert, agree: I'm game.

Throughout history humans have endeavored to communicate their thoughts and feelings through an ever evolving, infinitely multiplying succession of abstractions and representations—in a word, media. From these media have sprung a panoply of social sciences that seek to plumb the social, psychological, and emotional meaning embedded within them. Occasionally, the need to invent an entirely new medium arises when no other will do. Information visualization appears to be one such medium, and it fits comfortably on our expanding communication continuum.

Media have long been the darlings of psychological investigation: print, television, film, and more recently, the Internet have received deep, exploratory research. Vast amounts of research into the realms of cognition and mental imagery exist as well. However, design, as it relates to perception, cognition, and media has not been so lucky. While the visual portrayal of data is not new, widely available access to dynamic real-time data with which to create visual portrayals is. A sprinkling of studies concerning the design and cognitive processing of graphs, diagrams, and moving images (video and film) have been written but few peer-reviewed journal studies concerning design, cognition, and visualization (both static and dynamic) exist. A portion of the literature canvassing design and cognition lacks scientific rigor, is insensitive to the cognitive significance of design, or ignorant of the perceptual value of aesthetics, and some of the research is merely anecdotal. Consequently, in order to address the value of information visualization’s role in cognition and emotion, it’s necessary to cobble together different theories and concepts. Accordingly, the conclusions reached here are more associative than direct or explicit. All in all, it's a mixed bag.

Visualizations sit at the nexus of several independently fascinating fields—psychology, programming, science, and information design. Given the complex and layered communicative nature of information visualization, its role as a useful cognitive aid seems obvious; this in turn would seem to argue for immense and profound scientific interest in it. Yet, surprisingly, this is not the case—very little cognitive or affective research exists in this field. Perhaps research lags because of the relatively recent growth of visualizations as a communication medium on the Internet—a still nascent mainstream media phenomenon itself. Nonetheless, the dearth of psychological investigation in so important an area as information visualization, essentially a new visual language, must be corrected.

 

Copyright © 2011 Carla Casilli. All rights reserved.

In a youtube video, nobel prize winner Elinor Ostrom (2010) distinguishes between three easily confused concepts: 1) commons, "a wide diversity of private goods" that may include public goods like knowledge; 2) common pool resources, "a technical term [used] to refer to resources where it is difficult to exclude people" also to indicate that resource removal is universally decremental; and 3) the commons that includes "both public goods and common pool resources."

In his book Common as Air, Lewis Hyde differentiates between the commons and what he terms "unmanaged common-pool resources." Using England as a representative historical antecedent of today's commons, Hyde (2010) takes pains to note that true English commons were stinted with communal requirements for both use and contribution. In considering property ownership, he writes that its definition is rooted in the physical manifestation of land "plus the social relations and the traditional organizations that govern its use" (p. 29). For without the stint there is no true commons, but "only things belonging to no one, or pools of resources no one manages" (p. 44).

Hyde (2010) suggests three successive eras led to today's commons: 1) the Saxon age in which the majority of land was held and worked in common by villagers; 2) the post-Norman conquest age in which land typically was associated with a manor house and its attendant lord; and 3) the age of enclosure which heralded the modern concept of private ownership, and "ran from the early eighteenth century to the end of the nineteenth century" (p. 29).

Each successive era represents significant political and social change that affected and was affected by evolving notions of the commons. In moving from a commons to enclosure, accepted local customs were superceded by written national law. Additionally, the commodification of land significantly rebalanced social relationships by eliminating common shared resources, their associated activities and benefits, while introducing the concept of monetary valuation. This change effectively severed a village's connective tissue: once society members no longer needed to work together to maintain a commons, social obligations shifted from required to optional.

Societies offer a specific kind of value that modern citizens make use of everyday. But from whence does this value arise, for as Hyde asks, "…what exactly is 'property'? The oil in the lamp, the light it sheds, the midnight scholar's flash of insight: can each of these be 'property' and if so, by what ample definition of the term?" (p. 5). Together the combination of property and ownership produces the emergent aspect of value. Does the value lie in the outcome? What about the value of the generative components? And how do we decide what financial value air or water has? Or what the value of collective wisdom might be?

The gift economy of the indigenous people's Potlach rearranges the defining aspects of capitalism in a way similar to how cubism reimagined representational art. In this type of economy, "status is accorded to those who give the most to others" (Pinchot, 1995). Thanks to our seemingly innate reciprocity, gift-giving is self-reinforcing (as both Farmville's designers and players well know).

So, is the Internet a gift economy or a commons (as defined by both Ostrom and Hyde) arising from a loose and informal accretion of public knowledge? What do we want it to be? As vested political interests wrestle with this protean question, we'll soon find out.

To this I say, Citizen, arise!

References:

Collins, K. (Photographer). (2005, December 31). Fence [Photograph]. Retrieved from http://www.flickr.com/photos/kevincollins/81418404/sizes/m/

Hyde, L. (2010). Common as air. New York, NY: Farrar, Straus and Giroux.

IascCommons. (2010, August 10). Professor Elinor Ostrom interview - 01 [Video file]. Retrieved from http://www.youtube.com/watch?v=aXzbcgj9F54

Pinchot, G. (1995, 1997). The gift economy. In Context, 41 (49). Retrieved from http://www.context.org/ICLIB/IC41/PinchotG.htm.>

All possible valences of an object, all its ambivalence, which cannot be reduced to any model, are reduced by design to two rational components, two general models—utility and the aesthetic—which design isolates and artificially opposes to one another (Baudrillard, 1981).

Natalie Portman is a lovely girl; there's little to dispute there. However I find that her acting is, how shall I put this, not exactly the most believable. Nearly every time I find myself watching a movie with her in it, I think to myself, "How many other beautiful women are out there who can act? Why didn't the director choose one of them? How much more would I like this film with some other actress in it?" In other words, her luminous beauty isn't powerful enough to distract me from what what she's supposed to be doing—acting.

As I'm watching her I'm not thinking that she's a real character experiencing events that just happen to be on film. No, instead I'm thinking, this is Natalie Portman acting. This represents a vast and important difference. I'm calling this affective transportation/non-transportation difference the Natalie Portman effect.* And it's everywhere in the field of design, particularly interface design. Relying on the attractive to smooth over the ineffective.

We're pretty sure we know what we're getting when we see Ms. Portman's name on a movie marquis: she's tremendously attractive and we're used to seeing her in films. Because she offers us these few choice qualities, and we can mentally route around her emotional translation inefficiencies, we simply do so. Likewise, because producers know that she has an existing audience drawn in by her looks, they continue to select her to star in films. On paper she's a veritable win-win. So why am I complaining and what does this have to do with design? Good question.

Mentally step back. Imagine if we could slot some other doe-eyed ingenue who could believably and persuasively emote into one of Ms. Portman's roles. Someone who could not only visual entrance us, but also emotionally transport us. Imagine if we weren't locked into maximizing our initial investment with as little financial risk as possible. Imagine if we could imagine something entirely different. Then we'd be onto something.

Now take this same idea and translate it to design. If you're an interaction designer, imagine trying an entirely different and more useful interaction that doesn't rely on looks to get by; if you're a game designer, imagine if your game didn't rely on established themes or visual tropes; if you're a product designer, imagine if you could alter your product to last forever. What if you used a video to say what you wanted to say rather than relying on text? What if you provided everything as a podcast? What if you allowed people to leave audio comments rather than text comments? What if you stopped thinking of ways to shift your design towards an attractive norm and started thinking of ways that could radically improve it?

What if the expected norm wasn't the norm? What then?
update 02.13.11: What if we considered the contextualized solution the defining aspect of success rather than how closely it hews to the expected but unexamined norm?

I know that Jaron Lanier has taken some serious shit with regards to his writing style and ability to craft intelligible arguments, but you can't fault the guy for underscoring what he defines as digital collectivism and what I refer to as a series of software cages. Some of us are nicely kitted out in gilded ones, but many of us don't even know we're in one. If this is the world we choose for ourselves, and this increasingly appears to be so, then let's build it out in ways that are creative, dynamic, and beautifully functional. Let's heed the form and function concept and make sure to add the emotional component so often lacking. Let's not rely on the normative appearance of our work so that we can overlook its intrinsic and flawed affective emptiness. Let's move beyond the Natalie Portman effect.

References:

Baudrillard, J. (1981). Design and environment or how political economy escalates into cyberblitz in For a critique of the political economy of the sign, p 188-189. St Louis, MO: Telos Press.

Sirota, P. (2010) [Photographer]. Image of Natalie Portman for Marie Claire. downloaded from http://www.natalieportman.com/npcom.asp?page_number=176&parentId=151830

 

* I've called it the Natalie Portman effect but feel free to refer to this effect by substituting any extremely attractive but not exactly talented actor/actress of your own choosing.