Cognitive Science of How We Process Images & Words
This blog is part of a series of conversations between Busch System’s Senior Advisor, Alec Cooley and Brenda Pulley exploring the science of recycling behavior in relation to bin and signage design. If you missed it, you can find PART ONE of the series here, or you can continue on to PART THREE of the series here.
Note to reader: the first reference to a given academic study in this blog identifies them by their full name, with all subsequent references abbreviated to the lead author’s name with “et al.” and the publication year.
Alec: We’re back with this second of a three-blog series investigating the relative merits of using words versus images on recycling and waste bin labels and signage. If you’re just tuning in, you’ll want to go back to the first blog to read our synopsis of four different research studies that explored this question. Beware, things get a little dramatic at points.
Brenda: Yes, I got a little carried away at the end.
Alec: Perfectly natural. Bin signage brings out passionate feelings in all of us.
High level, the research we reviewed mostly but not universally seemed to endorse images as the more effective option to influence correct sorting between trash and recycling or compost bins.
Brenda: Yes, with emphasis on the “not universally”. And before we draw conclusions I think we need to better understand how people comprehend and relate to images and words. The studies we looked at simply quantified the impact to people’s sorting choices with different sign conditions. But what is it about photos or icons or words that’s actually prompting people to respond differently to one versus the others?
Alec: To do that, I think we need to pop under the hood, as it were, to better understand how the brain reacts to these. Let’s start with a common assumption we mentioned in part 1, that people simply process images faster. Well, turns out it’s true! While an oft-cited statistic about the brain processing images “60,000 times faster than words” appears to be more internet legend than hard fact, there’s plenty of science to back up the basic speed advantage of images. And that matters in the world of bin signage, because they have to perform outside of the lab in the real world where recycling and waste bins must compete for people’s attention. Someone walking down the street may only give a cursory glance at a set of bins, leaving a preciously small window of opportunity for signage to convey critical information.
Brenda: You like to refer to this as the “Homer Principle”.
Alec: Donuts, Duff beer or pretty much anything else someone can possibly think about will likely be more compelling and command more of their attention than handling a waste item that, by definition, holds no value to them.
Brenda: If you recall, one of the papers we reviewed in part 1, Wu, Lenkic, DiGiacomo, Cech, Zhao, Kingstone (2018) makes a point not just to rate the effectiveness of words and images on the basis of accurate sorting, but also calculated an “efficiency” score using a hybrid ratio of both accuracy and speed. I think this is important because, to your point, accuracy in a lab, where participants are free of distractions and time constraints have the luxury to focus their attention on the message displayed, doesn’t represent the real-world conditions signage has to function in. It’s measuring a theoretical accuracy.
Alec: And yet at the same time, simply going with signage that people respond to fastest misses the point entirely. The goal isn’t to get people to toss items in the wrong bin faster.
Brenda: Nope. A person may be quick to register what item is represented by an image or word, but there’s a separate cognitive step to translate that into meaning… what they should do it. And that’s where things get more complex.
Alec: These questions about signage extend beyond the recycling world, obviously. There are entire conferences dedicated just to designing wayfinding signage for example. And as you might expect, scores of research studies have explored the characteristics of effective traffic signage.
Brenda: Well, I know a few airports that could benefit from attending some such conference!
But back to Wu, et al. (2018), which by the way is a good launching point connecting this larger field of study to a recycling context. The paper’s introduction includes a review of several non-recycling focused studies that explore the underlying science of how the brain processes words and images differently. As we discovered, this is a pretty deep rabbit hole, but we’ll do our best to distill some of the key takeaways.
Alec: Warning to readers: if you simply want to know what to put on your signs, skip ahead to the final part three installment of this blog where we offer our advice. For everyone else, buckle up.
Images Depict Items, Words Represent Categories
Brenda: Wu, et al. (2018) summarizes a couple established points from the body of research. First, images aren’t simply faster to comprehend, they convey large amounts of information much more efficiently than words. Another paper we reviewed, Amit, Algom & Trope (2009), explains this further by observing that while images narrowly represent a specific item, words represent entire categories. They’re inherently inclusive of many items and only communicate specificity as you add modifying words. The word “building” by itself refers to an endless number of possible structures. Stringing together additional words narrows the field to, say, a “modern ten-story office building with a green tinted glass exterior, surrounded by older brick buildings”. You can keep going, filling an entire page of descriptive words, or, of course, you could simply show a picture.
Alec: You could argue that a picture, in a sense, has the same value as a dozen words to communicate a subject.
Brenda: Heck, they’re worth hundreds of words. Maybe even a thousand???
Alec: You’re absolutely right! Fascinating…
Brenda: If recycling only involved calling out one very specific item, say a clear 20-ounce #1 PET plastic water bottles, it would be simple to just show an image and be confident 99% of people will “efficiently” comprehend and know where to deposit the clear 20-ounce #1 PET plastic water bottle in their hand. But it’s obviously much more complex. Recall how the Purdue University study (Reimer, Roland & Banerji (2018) we discussed in an earlier blog identified nearly 450 distinct types of waste items during waste audits of a mixed academic and office building. With so many types of items people might be tossing out, and adding images for each gets unwieldly after the third or fourth one.
Brenda: Hold on, can’t we assume people are smart enough to know a picture of a clear 20 ounce #1 PET plastic water bottle is meant to imply other types of bottles?
Alec: Maybe, maybe not. If the word “bottle” by itself is ambiguous, how do you expect people to know which other types of bottles are implied by an image when it’s not even established in their minds that it should apply to anything other than the specific item depicted? Especially with our patch quilt system of subtle and sometimes radically different definitions of what’s acceptable from one location to the next. There’s too much left to subjective interpretation with a system that itself is too complex.
Brenda: And in fact that’s arguably what the results of the George Washington University / Keep America Beautiful Survey (2014) we reviewed in part 1 show. Remember, most people reacted to the word “Bottle” by assuming it included both plastic and glass bottles. But when they were shown an image of a plastic bottle by itself a majority assumed that meant glass bottles were excluded. Switching between words and images led to a similar result with different types of paper.
Alec: People’s experience and expectations are definitely a factor with all these results, and we should talk about the role of recycling education / outreach at some point. But it’s also fair to say personality types are involved. Some people are more likely to react to images with a literal, black and white interpretation where others may take flights of fancy assuming it extends to all sorts of items that may or may not be intended.
Brenda: Ok, let’s park this and head back down the rabbit hole…
Would you include a glass bottle with this sign?
What about this one?
Evaluating Images: Complexity, Familiarity and Concreteness
Brenda: So Wu, et al.(2018) points to another study, McDougall, Reppa, Kulik, & Taylor (2016)…
Alec: …not to be confused with McDougall, Curry & Bruijn (1999), McDougall, Curry & Bruijn (2000), McDougall et al. (2006), McDougall and Isherwood (2009), or for that matter, Isherwood and McDougall (2007).
Brenda: Absolutely not. McDougall, et al. (2016) as we’ll call it, notes how there are three basic characteristics researchers typically use to evaluate the effectiveness of images: complexity, familiarity and concreteness. The more visually complex an image is the more time it takes people to respond to it. Remember, comprehending the meaning or significance of an image is a separate step from simply recognizing it. A stylized icon of a bottle is pretty simple to absorb and respond to. Displaying multiple items or adding detail to the depictions adds complexity and slows down that cognitive process.
This isn’t rocket science, but familiarity also has a bearing on the speed and accuracy with which someone recognizes and grasps the context it represents. This is where the importance of advance education and clear, simple definitions of recyclability come into play. Because familiarity is about the ability to quickly access the appropriate meaning, and that can’t be learned on the fly. McDougall and Isherwood (2009) suggests familiarity is the most important determinant of the ease with which people identify an image.
Alec: Wait, did McDougall and Isherwood (2009) say that or was it Isherwood and McDougall (2007)?
Brenda: Both actually.
Alec: But not McDougall, et al. (2000)?
Brenda: No. McDougall, et al. (2000) and McDougall, et al. (2006) stayed on the sidelines for this one.
Alec: Sorry, not get hung up, but what about McDougall and McDougall (2013)?
Brenda: There is no “McDougall and McDougall (2013)”. You just made that up.
Alec: Just keeping you on your toes. Continue…
Brenda: There’s more to say about familiarity, but we’ll come back to that. Concreteness has to do with just how well an image actually depicts the item it’s intended to represent. Picasso named one of his paintings “The Weeping Woman”, but the average person can be forgiven if they struggle to confirm the subject matter with any speed, or frankly, accuracy.
Alec: I think this is important to note because a simplistic comparison of words vs. images glosses over the fact that not all images are equal. I’ve seen recycling signs with crudely drawn icons that would leave Picasso in tears.
Attention to detail, people!
Brenda: Good point. Poorly depicted icons are less concrete and as a result squander the efficiency advantage they’d otherwise have over words. McDougall, Curry & Bruijn (1999) lays out a whole framework to systematically measure and rate each of these image characteristics.
But, let’s jump back to Amit, et al. (2009) for a moment, because they go down a side tunnel on the idea of familiarity.
Alec: Yes, so Amit ran a series of experiments testing people’s responses to words and images based on the different ways we experience familiarity – cultural, temporal (distance of time) & spatial. They started with the hypothesis that because images convey lots of detailed information efficiently, they should perform well in situations where people have intimate knowledge of the items depicted. That intimacy or familiarity allows them to recognize and therefore capitalize on that large volume of information to react faster.
We flip the script with words. Because they represent broad categories of general information, they’re efficient at casting a wide net that, in theory, will perform better helping people with only a vague knowledge recognize they’re at least in the right ballpark. Images, on the other hand, shouldn’t work as well. The large volume of information they convey is simply lost on the person who doesn’t have a close relationship to the item depicted. They’re left to guess without the benefit of the wide net that words employ.
Brenda: And what did the study’s results show?
Alec: They pretty much confirmed their hunch.
Icons vs photographs
Brenda: There’s one last question to address. Is there a qualitative difference between photographs and icons, which are stylized drawings of items? Both get used on recycling and waste signage, but the choice of one over the other often seems arbitrary without consideration for performance.
Alec: I do think in recent years there’s been a trend amongst sign aficionados in favor of photos. Recycle Across America has certainly gone that direction with their decals. But to your point, I think a lot of this trend is based on gut-level hunch. What does the science say?
Brenda: Two of the recycling papers made a point to test their performance in separate sign conditions.
Boiling these down, Shahri, Vandenbergh & Samuels (2018) found that photographs led to greater sorting accuracy overall, and specifically with the paper and plastics recycling stream. Not with the trash, however, where accuracy was substantially the same. In Wu, et al. (2018) photographs prompted a faster response, which gave them an edge on the efficiency score (ratio of accuracy and speed). But actual sorting accuracy was about the same between photographs, icons and words.
Both papers call out potentially related points. Wu, et al. conject that the greater level of concreteness pictures generally have over icons could give them an advantage in situations where the sorting task and signage are unfamiliar. That would seem to help explain an observation Shahri et al. specifically calls out, that photos outperformed icons with the subset of participants who self-identified as non or only sometimes-recyclers. It’s not a stretch to say that people who recycle infrequently will be less familiar with the mechanics involved.
Alec: No, it isn’t. And I can think of other situations where even experienced recyclers will find themselves with less familiar sorting situations.
So what about the cognition studies? I’m interested to know how people process icons different from photos.
Brenda: I am too. But believe it or not, we didn’t come across any papers in our search that directly addressed this question. Seems a little surprising, huh?
Alec: Yah, it is. I don’t know if this could be called a theory or just a hunch, but if the high level of concreteness inherent to photographs makes them good at communicating a very specific item, and words are better for conveying broad categories, is it reasonable to think icons have the potential to straddle both to some degree? An icon image of a bottle will exhibit a greater degree of concreteness than the word “bottle”, but its stylized nature should, in theory, leave enough interpretive room for someone in the GWU / KAB study to assume it include a water, soda or even beer bottle. So while Shahri et al. and Wu, et al. showed the advantages of photographs… again this is the “Alec theory”… there may be situations where people’s level of knowledge, the complexities of the waste stream and the sorting guidelines converge to make photographs the preferred format. But change the levels of knowledge, complexity, etc., and there could be situations where icons are the way to go.
Brenda: Theory or hunch, that might be the case. I have a half dozen of my own theories on this, but what’s clear is this is pretty complex stuff. It would be great to see what other research is out there or that may come in the future.
Closing
Alec: Ok, final, final question: McDougall and Isherwood (2009) and Isherwood and McDougall (2007). Why change the order of the names? Wouldn’t it be simpler to stick with McDougall and Isherwood in both cases?
Brenda: Oh, look at that! It’s 5 o’clock already. I’ve got a bus to catch.
Alec: Huh? 5 o’clock in England maybe.
Brenda: Good enough for me!
Alec: I guess we’ll have to save this for part 3 of this blog series.
Brenda: Or not. What we will do in part 3 is apply some of these concepts to a recycling context and offer our final takeaway’s on how they inform the design of effective signage.
Alec: Ok, scoot along. I don’t want to you missing the bus on my account.
Brenda: Ciao!
****
Read PART THREE summarizing the takeaways and recommendations for designing signage.
Missed PART ONE of The Use of Images vs. Words on Recycling Signage series? Click here.
