Boxed risk warnings: research findings

by Ruth Shrensky

editor's note: This review of the research on warnings was undertaken as part of a larger project to develop a new edition of Writing about medicines for people, our guide for writers of Consumer Medicines Information (CMI). Warning messages, particularly of the boxed variety, are popular with regulators in many jurisdictions in Australia, Europe, and North America, and we are under continuous pressure to introduce this type of warning into CMI. But the newer research does not support the use of this type of warning. If anything, it reinforces the reasons why such warnings are inappropriate.

We publish this review here in the hope that it will help those making decisions about how to regulate information for the public make sound decisions based on evidence rather than popular opinion.

Warnings do not often work

A contradiction exists between empirical evidence showing warning labels to have little or no value in reducing risk, and intuitive reasoning and legal constraints which may favor using warning labels. (McCarthy et al 1982:168)

A warning is any information that is intended to change behaviour and prevent accidents. But there is a large gap between what is intended and what actually happens:

In spite of the widespread use of warning labels, searches for scientific evidence have yielded virtually no reason to anticipate that warning labels on consumer products serve as effective mechanisms to increase safety. (McCarthy et al 1984: 162)

Scientific research on the efficacy of warnings of risk (in this context, scientific research means research under controlled laboratory conditions) began in the 1950s, greatly expanded in the 1980s due to the increasing number of product liability cases, and is now well-established (Wogalter 2006:18).

In general, the empirical research findings (in, for example, the papers and abstracts in Laughery et al 1994 and in many of the papers in Wogalter 2006) bear out the theoretical and conceptual studies carried out by sociologists, anthropologists, and designers (for example, papers in Wogalter 2006 and research reports by CRIA 2006a, 2006b). They demonstrate that the effect of a warning message on a person's behaviour is unpredictable: although a warning can be noticed, read, and acted upon, it is just as likely to be missed, ignored, or have an effect opposite to the one intended. Compliance is rarely found above 50% of participants, and in some cases it is lower than 25%. As one researcher commented on effects of warning messages on labels:

It is unrealistic to expect warning labels to universally mediate safe behaviour in the user population (Gill et al 1987:52)

Legally, attaching a warning about the risks of using a consumer product is often mandatory; but it should not be enough to just provide the mandatory warning label, signpost, paragraph or graphic, then sit back and expect understanding and compliance. Nor is it enough to simply follow common beliefs about how to make warnings salient, such as making them bright red or framing them in a box.

On the contrary, studies demonstrate over and over again that this does not ensure compliance. Most of the papers in Laughery et al's (1994) collection demonstrate that risk warnings—whether on consumer products, labels, signposts or documents—are too often inadequate and ineffective, because their design does not take into account the many psychological and socio-psychological factors involved in producing an effective warning message. These include (and there are many others):

• the number of warnings a person is exposed to
• the past experience of the person
• the motivation and interest of the person
• the person's perception of the likelihood of the risk
• the person's attitude toward risk
• the perceived cost or effort involved in compliance (including actually reading the warning notice itself).

Most of the authors in Laughery et al are psychologists, engineers, failure analysts, or industrial designers. Studying the kind of factors listed above is what researchers in these disciplines do. The studies confirm that consumers often do not comply with warnings, identify up to a point which designs fail, and go part of the way in saying why they fail. The studies also pinpoint many of the factors that go towards explaining what happens when consumers are exposed to risk and to warnings about risk.

But useful and important as these studies are, they are limited. Firstly, many of the factors identified by the studies are unpredictable or non-predictable, certainly not the kind of things that can be accurately measured under controlled laboratory conditions by scientific researchers. Secondly, the studies do not adequately provide evidence on how to design warnings that actually work. Thirdly, in the words of one of the researchers:

...considerable attention should be paid to the design and placement of the warning [notice]. (Gill et al 1987:52, italics added)

But design and placement of signs are not the kinds of thing that research psychologists and failure analysts know about.

Ironically, these two factors can be measured and controlled, unlike the psychological and sociological factors listed above; but not by psychologists or industrial designers, since the design and placement of text are not psychological or socio-psychological phenomena. These factors are, however, understood and controlled by experts in a different field, namely information design; and information designers know how to exploit them to make warnings work better.

Making warnings work better

The skill of the information designer

Most of the papers in Laughery et al describe empirical research undertaken under laboratory conditions by psychologists and allied professionals. As noted above, these authorities can indicate what is wrong with a warning, but cannot begin to suggest what to do instead.

To do this entails two conditions: the first is research on actual warning messages in natural situations by genuine users in everyday contexts and environments (not research on specially created messages using controlled participants in the artificial conditions of a laboratory); and the second is research by experts well practiced in the design and placement of information texts and messages (not psychologists and experts in similar fields). What is needed, in fact, is diagnostic, observational testing undertaken by information or communication designers who can manipulate text and layout so that people find, read, and act on information and other texts, including warnings.

Information and communication design research has revealed the dynamic interplay not only between a reader's thoughts and feelings during reading (Schriver 1997), but also between a document and its reader (Sless passim). Schriver, Sless, Redish, Wright and others have spent many years undertaking both theoretical and empirical research into how people read documents, as well as implementing their findings in practice, resulting in evidence of improved communication between people. In other words, their research has led to practical solutions to the problem of designing documents that people can actually use. It has provided, and continues to develop, the means of improving communication between organisations and the general public.

Information designers describe the dynamic interplay between a document and a user as part of a dialogue—a 'conversation'—because written or graphic communication is similar to spoken in many respects: especially in that, if any kind of communication is to succeed, the actors involved have to take into account each other's characteristics (actual and perceived), agree on the purpose and context of their communication, and become involved in the negotiation of meanings.

A risk warning is part of a conversation, initiated by the producer of a goods or service; and this conversation, like any other, is only successful if the other parties participate and respond. This is beset by difficulties: one is, as Schriver notes, that while the other parties in this conversation know (or think they know) who the producer of the warning is, the producer does not know who the other parties are (apart from being abstracted 'consumers', 'clients', 'targets' and so on); another is that producers are beguiled by the seeming simplicity of a view of communication that states that all one has to do to communicate is to 'get your message across'. Although widely believed and followed, this is a simplistic view which accords with neither experience nor logic (Sless 1986; Shrensky 1994, 1996, 1998).

What's wrong with 'getting your message across'?

'Getting your message across' has a simple logic, called the 'hierarchy of effects': first the consumer or target must be exposed to the message, then in turn attend to it, understand it, form a positive attitude towards it, decide to act on it, and finally actually act on it; repeated reminders keeps the target remembering and acting on it (typical tables of the effects hierarchy can be found in McGuire 1989 and Backer et al 1992).

Experience has shown that this does not work. Think of the countless futile attempts to warn drivers about the risks of speeding , teenagers about the risks of drugs, and merry partygoers about the risks of driving home (many examples are given in Shrensky 1998, and Hugh MacKay's 1994 Why Don't People Listen? provides a more informal account of the inadequacy of the 'getting the message across' theory). Think of Nancy Reagan demanding 'Just say NO!!' or an exasperated parent yelling 'If I've told you once, I've told you a million times ...!!'

It would make little difference if you headed your warning with:

or

and put the wording of the warning in

big bold print

and placed it within

a black-bordered box

—these graphic devices may seem to you to make your warning noticeable to people, but it may not be so [1].

The research suggests that people choose for themselves what they will notice. Moreover, if you want a warning to be not only noticed but also read and acted upon reliably, consistently, and appropriately, you cannot just give the warning independently of its context, then expect the reader to read it and act upon it. As noted above, unless you engage in a conversation, you are ignoring the importance of the other partner in the communication act. This can be read as a lack of respect, and the result is failure. 'Getting your message across' is just the kind of disrespectful, one-way communication that results in inappropriate actions, as has been noted over and over again.

A conversation with a reader

In Consumer Medicine Information (CMI)—perhaps more than most commonly-used document—it may be tempting to believe that a prominent boxed-warning of a major, possibly fatal hazard, would be noticeable and acted upon. But research over many years has repeatedly demonstrated that boxed warnings set outside the main text may not even be noticeable, let alone acted upon.

A warning sign is only perceived as a warning- when it is part of the conversation between the document and its users. A warning must speak to the reader, taking into consideration:

• the context of the warning
• the placement of the warning
• the content of the warning

Context

The context of the warning includes who you are speaking to, the nature of the risk, what you would like people to do or refrain from doing, and alternative or preferred actions that are easily performed. A warning must make sense to people so they can weigh up the dangers of non-compliance with the ease of performing the preferred action. Typically, people are more than willing to accept risks, especially if:

• the risk seems unlikely
• the warning conflicts with experience
• the alternative requires a bothersome change of habit.

Moreover, there is no shortage of research demonstrating that unless a warning is immediately accompanied by the wherewithal to perform the alternative or by clear instructions on how to perform the alternative, it will often not have the desired effect (Dingus et al 1991a and 1991b). If, say, protective gloves are required when using a product, the warnings 'Gloves must be worn' or 'This product is corrosive' may not be effective, no matter how the warnings are presented, unless gloves are provided with the product. Similarly, a sign warning that a product may be affected by heat should not simply say 'Keep in cool place' but should specify 'Keep in [refrigerator, meat safe, larder, first aid box, medicine cabinet or whatever]'. (This is further considered under content.)

Placement

The placement of the warning can only be judged by observing where people actually look for information in a document. The evidence from CRIA document testing, across a broader range of documents than CMI, shows that information placed in a box and separated from the rest of the text will often be skipped over, not noticed ('filtered out') or ignored. See the illustrations below taken from two of our recent studies (CRIA 2006a, 2006b).

 

Information designers suggest several explanations for this:

1. there is no standard conventional meaning for boxed text. It can be used for:

• an 'aside', inessential to the rest of the text
• a different kind of text, such as an advertisement
• an extract 'pulled' from the text repeated in the body of the text
• an example or case study
• an important message.

Since there is no universally agreed convention, a designer cannot be sure that people will assume that what is in the box must be read, even if clearly marked 'Warning', since it is possible to skip over it without even noticing it. When the message is a warning about a possibly fatal risk, it is irresponsible to use such an uncertain device as a box set outside the main text.

2. people can ignore or totally miss a boxed warning sign unless they are actively looking for a boxed warning sign (Godfrey et al 1983; Strawbridge 1986; Desaulniers 1987; DeJoy 1989; Young 1991; Wogalter et al 1992; CRIA 2006a, b):

• an information document is not treated like a novel, read from start to finish: readers scan to find what they want, and read as little as possible (Frohlich 1986; Redish 1993; Schriver 1997)
• information should be in a clearly identifiable sequence with meaningful headings that make logical sense to the reader, so readers can easily find what they want; a warning in a box set apart from the rest of the text is not part of any logical sequence that makes sense to readers, so readers cannot hook it to the information they are looking for (this is further discussed under content)
• a warning in a box is usually just headed 'Warning' or the like; this is not useful, it has no content. What is it a warning of? Why should information seekers take the time to read it when it might not even be relevant to them?
• some people do not notice boxed warnings because they are on so many products they have become virtually invisible
• a warning set outside the text, headed 'Warning' etc is often interpreted as the manufacturer's way of avoiding litigation, otherwise not to be taken too seriously
• readers might catch sight of the word 'Warning' and think 'Yeah, yeah, hazardous to health if used improperly, I know, I know, b-o-oring!' and not bother to read the rest
• people do not look for warning messages if they do not perceive any great hazard.

These are only a few of the reasons why any warning must be carefully placed, and then tested for its performance. Information designers know that when items in a document are placed outside of a meaningful sequence (as they are in boxed warnings), they often have no impact. Information designers research, design, test and re-test a document until it has a sequence that readers can use and make sense of, including the appropriate placement of a warning.

Content

The content of the warning--that is, the information it contains and how this is written--must also be carefully considered (and tested) if you want the warning to be taken seriously and acted upon.

As noted above, users scan information documents to find what they want. If they know, or think they know, how to use a product, they may not look at the document until a problem occurs. Information design researchers who have observed users in action know that most users will quickly glance at a document to start with. If it looks disorganised, cluttered, wordy, or poorly printed, they may not use it; a user faced with a badly-written document may feel insulted, and lose respect for the producer (after all, the producer seems to have no respect for him or her). The document itself loses credibility.

But if on that first glance users can see that it is well ordered, with what they find to be instantly readable, logically-sequenced headings that have substantive content, they can see immediately what kind of information is present and in what ways the document can be helpful. And as long as the document has been tested by an experienced information design researcher, they can be assured that the information will be helpful.

It is here, in the text, as part of a logical sequence, under an appropriate heading, that a warning about a major risk should be placed (and preferably not in a box—see above). For instance, if the medicine manufacturer wants to draw particular attention to the risk to pregnant women, the warning should appear more than once in the CMI, under relevant CMI headings. It could appear under 'Before you take it', 'When you must not take it', 'How to take it', and possibly 'Side effects' as well. The warning could say:

If you are pregnant or intend to become pregnant, tell your doctor immediately. [Medicine] may affect your baby's development.

The probability that this will be noticed, read and acted upon is substantially higher, as it meaningfully answers the kinds of questions users ask, such as 'When should I take it?' 'How should I take it?', 'What might happen if I take it?'. Appearing more than once increases the likelihood that the user will see it, as well as emphasising its importance. It does not need the word 'Warning' or any other highlighting.

Conclusion

Understanding the risks of using medicines is essential if people are to be able to manage their own health. We must ensure the very best course of action in this matter, using evidence-based research and practice regarding placement of risk warnings where they are most likely to be noticed, read, taken seriously, and acted upon appropriately

The information design research described in this paper is based on direct observation. Schriver's studies of readers in action (for instance, teenagers reading brochures on the dangers of taking drugs or users reading instruction manuals) clearly demonstrate that readers' thoughts and feelings interact as they interpret documents, forming impressions and constructing ideas not only about the document but also about who is speaking to them through the document. She advises document designers to be very careful how they speak to their publics. Placing words inside a box without a useful context can be seen as talking out of turn and may be treated as such.

Redish's review (2000) on readability formulas (more than forty such formulas have been identified, including Plain English) points out the weaknesses of such formulas when compared with direct usability testing with actual readers; her well-known conclusions (1993)—that readers are reluctant to read more than they think they need—have often been confirmed and are now widely accepted by professional information designers. She advises document designers to ensure that important information is placed where it is expected and looked for. As the studies in Laughery et al repeatedly confirm, this is unlikely to be in a box outside the text.

Similarly the information designers at the Communication Research Institute of Australia have spent many years observing people using information documents—bills, maps, web sites, labels, consumer medicine information and so on—learning how people look for and read the information they need, and what they do with it. They advise that documents must be designed to match the way people read. More often than not, while readers regularly scan headings, they simply do not notice or read what is in boxes

Recommendations

• Respect the users of the document: place important material where it can be used.
• Remember always that boxed warnings placed outside the main text have scarce value, for many reasons.
• Use the conversational devices of context and sequence if you want to be taken seriously.
• Always be guided by an experienced information designer. Information designers have the experience and skills to place warnings where they are most likely to be seen and acted upon.
• Always rely on evidence of success, based on tests for each individual document; do not simply go along with popular belief.
• Remember that every document is different.http://www.communication.org.au/assets/images/end.gif” width="31" height="16" align="absmiddle">

References

Ayers Thomas J et al 1989. What is a warning and when will it work? Reprinted in Laughery et al (eds) 1994 op. cit, 1–5.

Backer Thomas E, Rogers Everett M, & Sopory Pradeep 1992. Designing Health Communication Campaigns: What Works? Newbury Pk: Sage.

Bliss James P & Fallon Corey K 2006. Active warnings: false alarms. In Wogalter (ed), op. cit 231–242.

CRIA 2006a. Research report for NSW Registry of Births Deaths and Marriages: Birth registration form. Melbourne: Communication Research Institute of Australia.

CRIA 2006b. Research report for CPA (Australia): Applicant information booklet. Melbourne: Communication Research Institute of Australia.

DeJoy David M 1989. Consumer product warnings: Review and analysis of effectiveness research. Reprinted in Laughery et al (eds) 1994 op. cit 164–168.

Desaulniers David R 1987. Layout, organisation, and the effectiveness of consumer product warnings. Reprinted in Laughery et al (eds)1994 op. cit 26–30.

Dingus Thomas A et al 1991a. A most critical warning variable: two demonstrations of the powerful effects of cost on warning compliance. Reprinted in Laughery et al (eds) 1994 op. cit 31–35.

Dingus Thomas A et al 1991b. Two reasons for providing protective equipment as part of hazardous consumer product packaging. Reprinted in Laughery et al (eds) 1994 op. cit 36–39.

Frohlich D M 1986. On the organisation of form-filling behaviour. Information design journal 5 (1) 43–59

Gill Richard T et al.1987 A comparative evaluation of warning label design. Reprinted in Laughery et al (eds) 1994 op. cit 50–52.

Godfrey Sandra J et al 1983. Warning messages: will the consumer bother to look? In Reprinted in Laughery et al (eds) 1994 op. cit 53–57.

Kalsher Michael J & Williams Kevin J 2006. Behavioral compliance: theory, methodology, and results. In Wogalter (ed) op. cit , p 315.

Laughery Kenneth R et al (eds) 1994. Human Factors Perspectives on Warnings: Selections from Human Factors and Ergonomics Society (HFES) Annual Meetings 1980–1993. Santa Monica, Calif: HFES

Mackay Hugh 1994. Why Don't People Listen? Solving the Communication Problem. Sydney: Pan.

McCarthey Roger L et al 1984. Product information presentation: presentation, user behaviour, and safety. Reprinted in Laughery et al (eds) 1994 op. cit 159–163.

McCarthy Roger L et al 1982. Warnings on consumer products: objective criteria for their use. Reprinted in Laughery et al (eds) 1994 op. cit 164–168.

McGuire William J 1989. Theoretical foundations of campaigns. In Rice Ronald & Atkin Charles K (eds) 1989. Public Communication Campaigns 2nd edition, 43–65. Newbury Park: Sage

Redish Janice C 1993. Understanding readers. In Barnum C & Carliner S (eds) Techniques for Technical Communication. New York: Macmillan, 15-41.

Redish Janice C 2000. Readability formulas have even more limitations than Klare discusses. Journal of Computer Documentation 24:3, 132–137.

Schriver Karen A 1997. Dynamics in Document Design. New York: John Wiley.

Shrensky Ruth 1994. Public communication campaigns. Communication News 7:2, 7–10. Canberra: Communication Research Institute of Australia. Reprinted in Sless David & Wiseman Rob (eds) 1996. Readings Towards Science Communication, 213–219. Canberra: Communication Research Press.

Shrensky Ruth 1996. Public communication campaigns—talking at or talking with? Proceedings of ANZCA Annual Conference, Queensland University of Technology, July 1996.

Shrensky Ruth 1998. The Ontology of Communication. PhD Thesis, University of Canberra.

Sless David 1986. Repairing messages: the hidden cost of inappropriate theory. Australian Journal of Communication 9 & 10, 85.

Sless David 1986–2006. Bibliography on request.

Strawbridge Jill A 1986. The influence of position, highlighting, and imbedding on warning effectiveness. Reprinted in Laughery et al 1994 (eds) op. cit 191–195.

Wogalter Michael S (ed) 2006. Handbook of Warnings. New Jersey: Lawrence Erlbaum.

Wogalter Michael S & Vigilante William J 2006, Attention switch and maintenance. In Wogalter (ed) 2006 op. cit , page 258

Wogalter Michael S et al 1992. The influence of location and pictorials on behavioral compliance to warnings. Reprinted in Laughery et al (eds) 1994 op. cit 164–168.

Wright P 1979. The quality control of document design. Information design journal 1 33–42.

Young Stephen L 1991. Increasing the noticeability of warnings: effects of pictorial, color, signal icon, and border. Reprinted in Laughery et al (eds) 1994 op. cit 164–168.

endnote

[1] Laughery & Young (1991:133) found that under laboratory conditions a boxed warning was unlikely to be noticed; indeed they found that a warning outside its context was noticed by a significant number of participants only when it featured not only a border but also a standard warning sign (exclamation mark in triangle), an iconic pictograph, and colour (red). To use all four features on a document such as a CMI would be impractical; more importantly, there is no evidence that increasing the salience of a sign improves compliance rates by any significant amount.