by David Sless and Robyn Penman
Review of: Simply Better: The case for clearer life office documents by David St L. Kelly & Christopher J. Balmford. Life Insurance Federation of Australia (LIFA) Inc. 1993. Originally published in Communication News 6(6) and 7(1) Nov/Dec 93 and Jan/Feb 94
In response to increasing pressure from government, consumer groups, and the media, the life insurance and superannuation industry has begun to focus on the quality of its documents. In 1992 the Life Insurance Federation of Australia (LIFA) established a Plain English Documentation Task Force to develop proposals for simplifying policy and policy-related documentation.
The Task Force has produced its first report. But how far does this report go towards addressing the community’s concerns about insurance documents? And how far does it provide a blueprint for the industry to follow? Other industries and government bodies—such as banking, pharmaceuticals, and legislative drafting—are under some pressure to improve the quality of their public communication. How far should they regard this document as a model to follow?
We have written this review for our members, particularly those in insurance companies, consumer groups, government, and the media, many of whom have a strong interest in the problems the report is trying to address. Can the insurance industry use the LIFA report as a basis for action, and as a means of assuaging its critics? Can consumer groups be confident that the solutions proposed in the report will genuinely meet their concerns? Can government be assured that the public interest is protected by this proposed self-regulation?
Self-regulation: some principles
The Task Force’s report is a proposal for self-regulation. We therefore begin by suggesting some principles which should be applied to judging proposals for self-regulation. We do this explicitly at the outset because we want to make clear the critical principles we are applying in this review. Also, we offer them so that there is a basis for discussion among interested parties.
We suggest the following principles. Self-regulations proposals should:
- draw from the most advanced practice and knowledge in the field
- be written by authors free of self-interest in the outcome
- be practical for the industry to apply, not placing unrealistic costs on the industry, restricting competition, or resulting in inefficient practices
- lead to improved industry practices
- allow for independent monitoring to satisfy both government and consumers
- be a model of the principles it advocates.
Good people on a flat earth
Because one of our primary objectives is to help government and industry improve the quality of its communication, we welcome the LIFA Task Force and similar initiatives. Beyond the obvious self-interest that motivates these initiatives, many involved are also motivated by a genuine concern for equity and openness, a desire for a fair go and good standards of corporate citizenship. This is most welcome; happily, we are living in a period when efficiency and equity are mutually supportive, and many people of good will have seized the opportunity to do good work. The best of intentions are, however, not enough to arrive at the best practical solutions (Sless 1990).
Best practical solutions only emerge when the most advanced knowledge is applied, and the LIFA initiative, like many others, fails to do this. The Task Force clearly understands that communication and document design are problematic. They clearly try to use research in communication to help guide them in their recommendations. But, because they are outsiders to the field, unaware of the significant developments in thinking and research that have occurred over the last fifty years, they tend to regard their common sense understanding of communication—getting a message across from one mind to another—as a sufficient basis for judging the relative value of particular types of research or research findings.
From our perspective within the field of communication research, we see the last fifty year’s research against an historical background of massive changes in assumptions and theory. We can see that not all the research methods or findings of the last fifty years have been equally valuable. For example, readability scores developed over twenty years ago made assumptions which, in the light of more recent work, are no longer tenable. We therefore regard readability scores as an interesting historical curiosity, but not useful as a research tool. By contrast, the LIFA Task Force—unaware of the underlying assumptions and their history—make extensive use of readability scores. We would argue that it is not sufficient simply to know what research methods or findings are available, one needs to know how to value and judge them.
An analogy from the physical sciences may help explain what we mean. Common sense derived from everyday practical experience suggested to prescientific societies that the earth was flat. This was not a self-consciously developed theory, just a description of how the world seemed, based on casual observation and common sense. It was only through systematic observation and measurement that people began to question this view, and advance the theory that the earth was round. Yet they still thought the earth was at the centre of the universe. From there it took a great deal more sophisticated measurement and argument to suggest that the earth was a minor planet orbiting a sun.
These massive changes in thinking and points of view are called paradigm shifts. Over the last fifty years there have been some paradigm shifts in our understanding of communication (Sless 1993a). We have moved a long way from believing in communication as a process of transmission—an equivalent of the flat earth view. But unlike the shifts in the physical sciences, the shifts in communication research have not yet had an impact on popular understanding which remains fixated on the idea of communication as a process of getting one’s message across from one mind to another. The authoritative Australian Government Style Manual in its latest edition claims that:
The purpose of good writing is to convey to another person what is in the writer’s mind. (p 3)
And well-meaning publications (Social Change Media 1994) published by the Commonwealth Department of Education Employment and Training as part of the adult literacy program, presents communication in the workplace as a kind of unproblematic pipe-cleaning exercise: all you need to do is follow some simple steps and all will be well—communication is problematic, but only in a technical sense. According to this view, we can improve communication by applying some new techniques, but we don’t have to change our thinking about communication.
Not surprisingly, therefore, people coming to our field without a knowledge of its history use this idea of communication, unaware that there exist different, more useful, and altogether better points of view. It is as if a group of Flat-Earthers read research from all paradigms—their own point of view, a geocentric view, and a heliocentric view—valued them all equally, and tried to make sense of them all within a flat earth view.
The Task Force is acting with the best of intentions, trying to build a case for clearer communication within the insurance industry. But it is massively handicapped because its members lack the necessary critical abilities and historical perspective to value and judge different research methods or findings they cite.
In all research fields there is an inevitable time lag between the development of new paradigms—new ways of thinking—and their practical application. People persist in old ways of thinking regardless of their merit.
In the field of communication we face a double problem: not only is there a time lag between old and new, but the users of the old flat earth view are unaware of the fact that the new is not just more of the same: it is radically different in its assumptions and view of the universe.
The Task Force and its brief
There were 14 members on the Task Force, including representation from the Insurance and Superannuation Commission, the Trade Practices Commission and the Federal Bureau of Consumer Affairs—all bodies with a strong and long-standing interest in the issues facing the industry.
The Task Forces’ brief was as follows:
The task of simplifying consumer documents means the employment of principles of good communication and extends beyond the use of simple words. The task, therefore, is to develop proposals and standard wording which:
- Have regard to the consumer and the ease with which a document can be read and used, that is, its friendliness to the consumer.
- Easily imparts [sic] intended concepts and messages. The purpose of a document must guide its construction
- Enables [sic] consumers to make a considered rational decision on the information provided
- Use communication and design techniques for emphasis and understanding
- Have regard to any legal requirements. (pp ii-iii).
There is actually much of value in this statement, despite its confusions and anachronisms (and some poor grammar). It shows an industry willing to come to terms with a major problem and take account of consumer interests.
But because of its many confusions and anachronisms, it does not clearly spell out the principles that should be applied to judging insurance documents from a consumer point of view.
We suggest three things which consumers should be able to do with insurance documents:
- read and use the documents easily
- understand their content
- use them to make sensible decisions.
These objectives are all to be found in the above statement, but they are somewhat lost. If these are focused on, as we suggest, they lend themselves readily to observation and measurement, allowing for independent monitoring to satisfy both government and consumer. In other words, it is possible to test any particular insurance document to see whether it meets these performance objectives. As we have suggested, this is an important principle for any industry initiative in self-regulation. The principles it applies to judge its peers must be explicit and independently verifiable. We will return to this issue throughout this review.
Other matters raised in the statement are either vague, confused, inappropriate or subordinate to the main objectives.
The report begins with a brief background to the industry, its products, regulation, and finally the variety of documents which the industry uses in its relations with customers. Such an inventory is a necessary part of such reports, but these inventories must be complete. Unfortunately, the final section on documentation is incomplete. For example, no mention is made of standard letters—one of the most difficult and intractable problems facing insurance companies. No mention is made of personal statements, pursuit and health questionnaires, claim forms and forms to change the conditions of a policy. Moreover, no account of the documents would be complete without considering the way in which these documents interact with other types of consumer communication, such as with agents, counter staff, or telephone enquiry staff. This section fails to reveal the sheer scale and complexity of documentary transactions that go on between insurance companies and consumers.
Most insurance companies are themselves unaware of the scale and complexity of these transactions. And many lack the policy, skills or technology that would enable them to gain this awareness, let alone do something about it. In an industry groaning under the weight of its own paper production, we need a much better assessment of the scale of the problem before suggesting changes. Unless this is done, the industry will rightly be concerned that the proposal could lead to unrealistic costs.
Understanding insurance documents
In Chapter 2, the report provides a summary of some research done for the Trade Practices Commission on insurance and superannuation documents, which ‘prove’ that insurance documents are difficult to understand—as if proof were needed!
The studies commissioned by the Trade Practices Commission used readability scores and studies of document structure. But for a long time now, we have known that readability scores and studies of document structure do not predict whether documents can be read and used easily. These techniques do not provide an independently verifiable measure of a document’s performance, and therefore cannot be used to satisfy principles for self-regulation. The claim by the authors of the LIFA report that these measures are in some sense ‘objective’ is entirely spurious; they are the result of applying subjectively-derived categories in a highly subjective manner to written material. The fact that numbers come out at the end does not make the measures objective, and anyway it is not clear what they ‘measure’. Based on the ‘evidence’ offered by these studies, nothing can be claimed about insurance document comprehensibility. In fact, these studies add nothing to our stock of knowledge about consumer incomprehension of insurance documents, they merely illustrate how the application of an old flat earth paradigm wastes public money (Sless 1985).
The only true measure of a document’s comprehensibility is its comprehensibility by actual users. We can only judge whether a document is likely to be understood by users as a result of testing it with users. One of the great lessons of communication research is that our ability to predict anything is severely constrained by context and that testing in context is the only reliable measure we have of a document’s likely performance in real life. But we do not have to test every insurance document to prove that as a group of documents they are hard to use and understand.
Ironically, the evidence for the poor quality of insurance documents is to hand. We know quite a lot now about best practices in document design (Penman and Sless 1992). We also know that in the absence of these best practices (which include testing) most documents perform poorly: users have difficulties understanding and using them.
Best practices in document design require the presence of certain skills, and the application of certain procedures. A few phone calls to each insurance company would have revealed that in most of them the skills are absent and that best practice procedures are not followed.
Yet buried in Chapter 2 is a little gem. The only study commissioned by the TPC that offers some useful lessons in support of best practices is one in which Melbourne University students were asked for their opinions of some insurance documents after they had unsuccessfully used them to answer questions about insurance. The study showed that:
[Students] who rated the comprehensibility of the documents highly did so with a false sense of confidence. (p 10)
This finding is consistent with a great many other studies which show that people’s judgements about the comprehensibility of documents are unreliable (even if they are plain English experts). As this particular research was conducted by the authors of the report—both plain English advocates—we hope they benefit from the lesson it has to offer.
The above finding should give rise to a much deeper cause for consumer and government concern. Documents written in plain English and presented using good graphics seem superficially to be more comprehensible, and the research shows that they are preferred by people. But, as the above study and others confirm, people’s preferences are unrelated to their comprehension, and their confidence in the preferences is misplaced (eg Keller-Cohen et al 1990). There is a very real danger that consumers will be lulled into a false sense of confidence because a document looks well designed and is written in plain English. Consumers may think that they understand a document and know how to use it, when they do not. As a consequence, they may be seriously misled or disadvantaged.
This is not an argument against good-looking or well-written documents, but an argument in favour of much more stringent tests, when trying to establish a document’s comprehensibility, than the appearance of the document or people’s preference for it .
Chapter 2 of the report concludes with an attempt to explain why insurance documents are complex. The reasons given are those consistently offered by plain English advocates, namely that lawyers have learnt bad drafting habits and write for the courts rather than consumers. While this is undoubtedly true, it mistakes the symptoms for the cause. The reasons why these practices have persisted is because the lawyers and the courts have until recently held all the power: small print and arcane complex language is a manifestation of that power. The only reason why insurance companies are now changing is because consumer purchasing power and consequent consumer legislation have altered the balance of power in insurance transactions. Insurance companies are now dependent on consumer spending and confidence in a highly competitive market. And changes in both insurance and trades practices legislation have given consumers more protection and power within that marketplace. The current concern for better insurance documents is part of the re-alignment of interests in a struggle for control and power in the market. To see it as anything other than that is naive. Within this framework there are, of course, many people of good will who want to do good work, and in the current environment such work will flourish. But no one should be under any illusion that better insurance documents are a response to changed circumstances, not an instrument of change in their own right.
So lawyers will acquire better habits, and better manners. But they will only continue to exercise these habits if it remains in the interest of the insurance companies for this to happen. Consumer and government organisations concerned with advancing consumer power should be vigilant to ensure that changes to insurance documents are real and not superficial token exercises in plain English and layout. They should be careful to ensure that they do not mistake the symbols of their new-found power for reality. Documents must really be comprehensible, not just seem to be so.
Clear communication tautologies
Buried in the Task Force’s brief is a definition of clear communication:
Documents that clearly communicate are ones that can be read easily, can be understood, and allow people to make sensible decisions. Therefore it follows that adopting clear communication principles will necessarily lead to greater effectiveness, efficiency and competitiveness.
Just as the statement ‘This square has four sides’ tells us nothing new about squares, so most of Chapter 3, extolling the benefits of clear communication, tells us nothing new about clear communication or its benefits. It does not answer the question: how does one achieve clear communication?
The authors are, however, in no doubt that the answer to the question is plain English. Unfortunately the evidence they offer for this is unconvincing. They cite the ‘estimated’ savings made by the UK government redesigning and rationalising its forms, and our own work on insurance forms for Capita Financial Group (Fisher & Sless 1990). As readers of this newsletter will know, neither of these cases offers evidence for the efficacy of plain English. In fact the Capita study deliberately had nothing to do with plain English (Sless 1993b).
Clear communication & the law
Chapter 3 does provide a compelling legal case for clear communication, which has application in many areas both inside and outside the insurance industry. Indeed, given the legal credentials of the authors, this is the most important and authoritative part of the report and one that should be clearly understood by all industries who provide consumers with information or contracts relating to their products and services.
At the heart of the legal opinion offered by Kelly and Balmford is the doctrine of unconscionability. Put simply, it means that it is unreasonable to expect someone to abide by an agreement they don’t understand. In the past, signing an agreement like an insurance contract bound you to the terms of that agreement whether you understood them or not. However, courts have recently taken consumers’ understanding of documents into account in making decisions.
Moreover, as Kelly & Balmford note:
The doctrine of unconscionability is not just a judicial development. It is also to be found in Section 51AB of the Trades Practices Act 1974. … Should the Trade Practices Commission view [that 51AB applies to insurance contracts] be adopted and enacted, the insurance industry will be at further risk of having some of its documents made ineffective under Section 51AB. (p 16)
They go on to discuss the doctrine of ‘utmost good faith’ in the Insurance Contracts Act 1984 and conclude that Sections 13 and 14 of the Act may:
… pose an even greater threat to the effectiveness of some insurance documents than the doctrine of unconscionability. (p 17)
Nothing more clearly shows the shift in power from insurer to insured, and why insurance documents need to be more understandable. But despite this legal development in favour of consumers, it is not yet clear what constitutes evidence of understanding. Kelly and Balmford argue that if a document is in plain English it is necessarily understandable. We would argue that more stringent criteria are needed, since the presence of plain English is neither a necessary nor sufficient condition to make a document understandable (Penman 1992a). Only user testing provides the evidence that a document is understandable.
We are not alone in this view. At a recent seminar on Consumer Product Information (CPI) in the pharmaceutical industry, Bernard O’Shea, a distinguished industry lawyer, gave the opinion that it was a legal requirement for CPIs to be understood by consumers, and that testing was essential to determine that they were understandable.
Chapter 4, 5 and 6 try to deal with methods of improving and testing documents for comprehensibility. This is the weakest part of the report. There are numerous technical inaccuracies and misunderstandings of the research literature. The advice given is at times too general and at times at odds with best practices in the field.
Chapter 5 on testing is the most seriously deficient. Most of the methods discussed have no predictive validity, established reliability or sensitivity.
The authors place a great deal of emphasis on what they call ‘content testing’ which seems to come to a remarkably obvious conclusion: if particular information is not in the document, the document will be in breach of the relevant guidelines and requirements, and consumers will not be able to use information since it isn’t there. It seems to us that writers could deal with this problem with a simple checklist in front them.
The only method with some value is what the authors call ‘client-based tests’—known more familiarly in plain English as user testing. Misquoting some of our research (Penman 1992b), the authors give no proper account of this technique which involves collaborating with readers throughout the development of a document. Unfortunately they regard testing as something one does at the end of document development, not throughout. Further, they erroneously suggest that this type of testing requires a high level of skill, only available from relatively expensive external consultants. In fact, properly applied, using internal resources, this method can be remarkably cheap. Done in the way they suggest it will lead to some very expensive mistakes. They go on to advise:
In some cases [user testing] can delay the introduction of new documents. (p 33)
These observations on testing reveal a serious misunderstanding of best practice and what is involved in the proper planning of document design projects. The report does not demonstrate best practice, far from it; in particular, the shortcomings of Chapters 4, 5 and 6 are typically those made by people with little industrial experience who are still learning the craft of document design.
The total absence of any detailed costings and cost benefit analysis should also be of serious concern to industry.
Perhaps the greatest single weakness of this report is that it offers no independent evidence by which the community might judge improved industry practices. Surely this must be the central confidence-building evidence that will allow government and consumer groups to trust the industry to regulate its own affairs. Without an independent yardstick all we can have is the utmost good faith that the industry will begin to effectively regulate its behaviour in an area where it has never done so before. Some members of the community may want more than faith.
Fisher, P. & Sless, D. (1990) Improving information management in the insurance industry: A case study of Capita Financial Group. CRIA Occasional Paper 10. Communication Research Institute of Australia, Canberra.
Keller-Cohen, D., Meader, B.I., & Mann, D.W. (1990). Redesigning a telephone bill. Information Design Journal 6 (1), 45–66.
Kelly, D. & Balmford, C. (1993) Simply Better: The case for clearer Life office documents. Life Insurance Federation of Australia.
Penman, R. (1992a). Plain English: Wrong solution to an important problem. Australian Journal of Communication 19 (3), 1–18.
Penman, R. (1992b). Well-designed documents are important: But what is a well-designed document? Invited address to LIFA conference, Sydney, Sept. 1–2.
Penman, R. & Sless, D. (1992). Designing information for people. Communication Research Press, Canberra.
Sless, D. (1985). Repairing messages: The hidden cost of inappropriate theory. Occasional Paper No. 1. Canberra: Communication Research Institute of Australia.
Sless, D. (1990). Equity and efficiency in corporate communication: The emerging challenge. Australian Journal of Communication 17(1), 19–42.
Sless, D. (1993a). Inside communication research. Communication News 6(2), 5–7.
Sless, D. (1993b). Plain English stories. Communication News 6(5), 1–3.
Social Change Media (1994). Communicating for Success. Commonwealth Department of Employment Education and Training, Canberra.