Book Review:
The Nurnberg Funnel
by John M. Carroll

Reviewed by
Robert E. Horn
Chairman, Information Mapping, Inc.
in The Journal of Computer Documentation

The frontpiece of this book shows a plump, happy faced youth dressed in the clothes of several hundred years ago with a funnel sticking out of the top of his head. Two teachers are industriously pouring large amounts of liquid knowledge into his head. This is the Nurnberg Funnel. It was supposed to "make people wise quickly." But the author of this book says, "There is not and never was a Nurnberg Funnel." (p.102) "Nevertheless the need to make people wise very quickly is often overwhelming. And in trying to make instruction efficient, designers have frequently lapses into trying to pour information into the learner's mind."

Hundreds of thousands of software packages come into offices every year. People like you and me have to learn how to use them. It has gotten a little easier in the last ten years. But it is still an uphill climb. Why? In large part because the mental models that programmers have of users is not generally as sophisticated as their coding techniques. And we continue to suffer because interface design is an underdeveloped skill.

What about the training and documentation that is supposed to help us? Its quality ranges from extremely poor to excellent, with a weighting toward the poor side. In the Nurnberg Funnel: Designing Minimalist Instruction (Cambridge, MIT Press, 1991), John Carroll presents some helpful ideas based on some useful research on how the initial self-instruction (often called "tutorials") should be developed and written.

Carroll's Critique of Systematic Self-Instruction
Caroll thinks that "standard self-instruction" is often in "conflict with the learning styles and strategies" that ordinary people adopt spontaneously. These are the tutorials which "required users to proceed step by step through sequences of drill and practice exercises." (p.5) He has strong words to say about what he calls the standard systems approach to instruction. He says it has fundamental flaws and regards it as a single, thin, mechanical approach which only superficially understands how human learning takes place. In fact, for Carroll, systematic approach to instruction is the Bad "S-Word" pervading the otherwise happy world of the learner. Carroll's indictment, if substantiated, extends to much of the computer-based training approach being implemented these days. There is no questions that many CBT courses are numbingly dull drill and practice. On the other hand, it is a hopeful sign that somebody within the world's largest computer company has finally noticed how bad their manuals generally are. One is unable to determine from Carroll's exposition, however, if the S-word writers were represented in Carroll's experiments by the best of their genre or the worst.

Perhaps this is a good time to mention my bias. I am most frequently placed in the S-word category on the basis of the best-known of my work, the invention of a structured writing methodology of considerable systematicity. (see Horn, 1989 for a summary) But, lest you think that I am overly biased in that direction, I want to mention that a major influence on the development of my approach was directly as a result of experiments in learner controlled sequencing of instruction. (The import of these remarks will become relevant as I describe the fundamental place of "guided discovery" in Carroll's educational philosophy.) In fact, for several years (1967-71) our research was focused on how to produce a computer-based manuals and instruction that could be either system-controlled or learner-controlled. I can only say that I have strong biases in both directions.

The Problems of Initial Learners
Many of the learners Carroll studied "do not follow directions willingly or well."(p.26) They "made many kinds of errors in following seemingly clear instructions in the manuals. They typically created and responded to their own agenda of goals and concerns, not to the careful ordering of steps in a training procedure. They jumped in, skipped around, forgot about the manual for a while, consulting it when the page it was open to no longer corresponded with the current system state" Furthermore "Learners...don't seem to appreciate overviews, reviews, and previews, they want to do their work. They come to the learning task with a personal agenda of goals and concerns that can structure their use of training materials."(p.26) And they had "difficulties learning from rote exercises even when they do successfully follow the directions." (p.32)

In short, Carroll concludes: "These (the S-word tutorial) manuals often do not make provisions for learners to take initiative to learn what they want, when they want, nor do they provide support for the kinds of problems such initiative can produce."(p.133) The confusion in such a statement in part is Carroll's not successfully spelling out in the book his philosophy of reference manuals a companion to tutorials. I have been uncomfortable with making such a large cleavage, such an enormous "either-or" out of the continuum between guided exploration and systematic instruction. A better design attempt would be to try to get the best of both worlds, if possible.

The Minimalist Approach
Carroll has a different approach which he calls "minimalist." And in this book, Carroll contributes a good deal of solid analysis and creativity to the problem of initial learning of tangled, knotty, complicated, unforgiving computer systems. "The key idea in the minimalist approach is to present the smallest possible obstacle to learners' efforts, to accommodate, even to exploit, the learning strategies that cause problems for learners using systematic instructional materials. The goals is to let the learner get more out of the training experience by providing less overt training structure." (p. 77-78) One might be tempted to think that it resembles a guided hunt and peck method of learning to type, but it is a lot more than that. His design approach for initial computer training is encapsulated in nine principles (which I paraphrase):

1. Use real tasks for the training exercises and let users select their own tasks. It enables people to use their prerequisite competence and engages a "powerful source of motivation." Not a new principle, but one which Carroll demonstrates the value of extremely well.

2. Get the learner started on real tasks fast by eliminating almost all front-end orientational material. Extensive preambles can "obstruct meaningful activity." Excellent advice.

3. Guide learners' reasoning, exploring and improvising with questions and other hints. Carroll sometimes recommends presenting incomplete training materials, so that learners have to explore. This is a core principle, because it directs the focus of learning activity and provides the most contrast with many conventional approaches. Carroll makes a good case for this principle. He also suggests presenting summaries in place of complete texts. This sounds a lot like providing what are commonly called job aids. More on this later.

4. Design the materials so that they can be read in any order in so far as possible. This principle permits learners to "support their own goal-directed activities." Use "high degree of modularity" and "small, self-contained units." As the originator of structured writing that introduced "information blocks" to replace paragraphs, and the author of a book on hypertext (Horn, 1989), you won't get me to disagree with this principle.

5. Help learners to coordinate training materials and software by providing landmarks for normal or error situations. Illustrations which show what the screen should look like if everything is OK are the primary example that Carroll gives of this principle. Again, good advice.

6. Focus early attention in the training materials on enabling the learner to recognize and recover from errors. Learners make many kinds of errors in learning computer systems. "Training materials must therefore explicitly support the recognition of and recovery from error both to make the materials robust with respect to user error and to train error recovery skills." (p.10) Guided exploration of error possibilities is important to speed up initial learning and decrease the frustration resulting from making so many errors. This focus on errors is Carroll's greatest contribution to the writing of initial learning manuals for computers. All computer training and documentation writers should read this book -- if only for its treatment of this topic. However, the book is just as important for computer programmers and interface designers who should ponder deeply the difficulties the vivid descriptions of the difficulties that users have with systems.

7. Engage the learner's prior knowledge in introducing novel concepts. Use familiar office tasks, language and metaphors. Highlight differences in operation of the system from what might be expected from the learner's background. Once more, a good principle, but not a particularly new one.

8. Consider using the learning situation, as opposed to practical on-the-job examples, for learning examples, exercises and explorations. Help the learner understand the "fine detail of the actual situations in order to create practical solutions." (p. 90) This is a tricky one to execute in practice. Carroll does a good job of it in the materials he describes. I usually do not recommend this approach and instead, suggest using on-the-job examples rather than difficulties encountered at the moment, because of the greater ease of writing those exercises and examples and because there is less chance of confusion between the instructions and the examples. But Carroll is justified in using in this principle especially when it is tightly coupled with his other ones.

9. Aim for optimizing learning designs by repeated testing and avoiding the temptation to systematize approaches into checklists. Carroll says, "There is no deductive theory of minimalist instruction; that is, given a set of minimalist principles, we cannot just crank out a training manual. Design never works this way." (p.91) It seems a little strange that we are thus prevented by this rule from generalizing from doing a series of manuals on the same class of manuals. I would imagine that a dozen word processing tutorials prepared by Carroll and colleagues would have some strong similarities, and that one might even be able to squeeze a few systematic pieces of general advice for developing them (beyond the nine just described). In fact, that is what will very likely happen despite Carroll's explicit distaste for any more systematizing of minimalism.

What can we say about the nine principles taken together? They provide good guidelines which, if followed with perseverance, should provide good tutorials. What does Carroll do with them? He presents an approach to instructional materials development which he characterizes as an "eclectic synthesis of available design elements (such as interface and instructional metaphors, techniques for coordinating learner attention, tasks that learners want to accomplish) marshaled to address instructional objectives and usability objectives...The analysis of instructional objectives is decompositional, as in systematic training design, but at a far coarser grain of analysis..." He calls for early informal testing (of the formative variety) to "collect rapidly as much detailed, diagnostic information as possible." (p.91-92) He warns against relying on experts because they probably won't be able to anticipate the difficulties of learners and their very subject matter expertise is likely to get in the way of their design approach. This is followed by summative testing of the quantitative performance type. This overall view is not too different from that which you would find in many training and documentation shops, except perhaps for his greater emphasis on testing and evaluation, which are all too often omitted in the rush to get the documentation out the door in time for product delivery.

Described at this level of abstraction, Carroll's tutorial sounds pretty similar to what the S-word people do. Certainly we could presume that even his task analysis might be very similar to those of the S-word people. But Carroll does not tell us much about his task analysis approach. And it is probably true that, as Carroll claims, many of the S-people err on the side of making the tasks too tiny. On the other hand, many among the S-word people have critiqued other S-practitioners excessively fine grained analysis. Task analysis is needed. And I would suggest that it had better be systematic. I might also mentions that formative evaluation, outside the psychology research laboratory, when practiced without explicit systematic goals can also be extremely limited, uninformative, and inefficient. And it can be very valuable when done well.

The Minimalist Manuals
Carroll shows us less than I would like to have seen of a complete minimalist tutorial. This is unfortunate for an approach that puts such strong emphasis on learning by case study. Twenty-five pages of a minimalist manual in the appendix would have helped us get a better fix on what Carroll has in mind. But he does present several isolated pages. From his text we gather that a typical word processing application would be comprised of a series of 25 one-page cards with approximately 100 to 200 words broken into six or seven chunks each with a different function or message. Ideographs are used to highlight chunks of information and some illustrations (e.g. of screen display) are also presented. Carroll does not go into sufficient detail in explicating what these chunks might be, if there is similarity among types on all of the cards, and other questions that a writer might like to know. The cards have labels at the top such as "How do I use these cards?" "What if I get stuck?" "Typing something," "Quitting work," etc.

Best Application Areas for Minimalist Manuals
Carroll has done a lot of good thinking and research on computer tutorials and interface design over the last decade. Thus, for the training and documentation community, it is important to see just where his insights might be applied and just as important, where not to employ the method. Caroll's claims for minimalism particularly focus on initial self-instruction particularly for general purpose software such as operating systems, wordprocessors, spreadsheets, time management and the like. These are the systems that provide the interactive context for working computers. The generalizability of his research may not extend to a lot of software packages on which much other processing takes place. I am not sure that his guided exploration approach would be what learners or managers would want to use for large amounts training in other areas. (e.g., Would you want the operators of the U.S. Treasury Dept T-Bills software to learn about the transactions concerning your investments with the minimalist guided exploration method?) This highlights an area that Carroll presumably leaves for future research, that is, common documentation management issues of how do you tell how much to train and how to determine cost-effectiveness of his approach in an operating environment. The area of tutorials for initial computer instruction is an area that has not had a lot of attention from training research. I, for one, am happy Carroll is working on it. It is equally important to notice what minimalism is not (necessarily) to be used for: advanced training and reference documentation. In some cases, Carroll himself gives his experimental subjects systematic reference documentation for advanced features of the software. The most advanced of these manuals appear to be done with good old S-word writing methods.

The Paradox of Sense Making
He places somewhat more emphasis than warranted on what he calls the paradox of sense making which he describes as the problem of people being "too busy learning to make much use of the instruction." (p.74) He asserts that people "can understand only through the effectiveness of their actions in the world..." and stopping to get instruction just interferes with this process. Thus, "The tension between personally meaningful interaction and guidance by a structured curriculum entails a priori limitations on how much we can ever accelerate learning." (p.93) He concludes, "There is no instantaneous step across the paradox of sense making. The project of minimalist instruction is to make a direct attempt to narrow this step...."(p.102)

Reference Functions of Documentation
Carroll's presentation is a little unbalanced in the sense that he fails to give sufficient acknowledgement to the necessary function of being able to look up rarely used, but important functions, in reference manuals. People do forget. In fact, we forget a great deal of what we learn. We are continually relearning stuff we used to know. That problem, it seems to me, calls for excellent reference documentation. I imagine many readers of this publication are tired of people saying, "I didn't read the documentation." Of course not! Reference documentation is not meant to be read from cover to cover. To his credit, Carroll keeps his focus narrowly on the first couple of days of learning of certain kinds of software systems.

Serialists and Holists
Pask (1976) describes two types of learners, "holists" and "serialist." Pask's description of holist learners sounds strikingly like those Carroll describes in the passages above (in the section titled "The Problems of Initial Learners."). One gathers from Carroll's book he believes almost all of us are holists. However, in my experience, roughly half of learners appear to be holists. Holists like to get the big picture before getting to the details. They like to start reading in the middle of books. They skip around a lot. They like making their own connections. Serialists on the on the other hand like to proceed through learning step by step. The like to fulfill all prerequisites necessary for one level before going to the next. They like to read books starting at the first page. Obviously, like many "personality typologies", it is better to regard "holist" and "serialist" as tendencies not types. Most people certainly can have aspects of both types. But Carroll might want to consider that some of his anomalous data on learner reactions to his minimalist manuals could well be explained by this distinction. I certainly found the serialist-holist distinction to explain otherwise puzzling data in some research that I have done on learning materials. Here are some of the quotes from Carroll that suggest serialists talking about Carroll's minimalist manuals: "...there was a tendency for learners to treat the intentionally incomplete hints, checkpoints and remedies as if they were the more familiar kind of step-by-step procedure."(p.136) Carroll also reports that several learners expressed a desire for "more structure" (p. 156) and that three participants "became frustrated enough to ask to be excused from the experiment and were replaced."(p. 167) Carroll is to be congratulated for reporting this anomalous data that runs counter to one of his major presuppositions (which is that substantially all learners act in the manner of holists) and he should not be blamed for failure to make something of it, since it would be hard to do that within the research and design paradigm he is working in.

Knowingly or unknowingly, we have all been attempting to design manuals for holist and serialist learners for many years. Carroll's approach will certainly appeal to the holist in us. More than that, he may even have at least part of a framework for a single manual for both. That remains to be seen. Carroll has done at least one fascinating research project that points in this direction. In this design he prepared four versions of a minimalist manual: skeletal, elaborative, inferential, and rehearsal. But, regretfully, he doesn't provide sufficient examples of these four versions for me to comment further.

Indexing
The importance of a good index is glossed over. Carroll says, "Indexes, for example, are a tool to help people read in any order by supporting structured search. However indexes often fail from lack of consensus about what terms should be indexed to support search...(and) readers often report that they prefer browsing to structured search." (p.84) Here we should avoid the "either-or" trap of browsing vs. indexes. Users will be much better served by a both-and approach, if the manual gets to any significant size.

Reinventing Job Aids
In one respect, Carroll appears to reinvent the wheel: the job aid. In fact a fair amount of his book sounds like it is a rationale for these handy training tools. This is one puzzling gap in Carroll's otherwise extraordinary depth and breadth of knowledge. For example, he seems to describe the development of simple job aids which have been well known and heavily used in the performance and training industry for at least 30 years. Yet this technology goes unmentioned, even though many of the principles and much of the approach he calls minimalist is quite similar or identical.

Training Wheels and Help
In the later parts of the book, Carroll reports what can be described as early exploratory research on restricting the functionality of software to prevent errors of initial learners (which he calls the "training wheels" method) some intriguing ideas for designing online help messages, and a theory that suggests that we will learn more about creating manuals and interfaces from repeated cycles of deep understanding of tasks, careful design of artifacts (e.g. software and interfaces) matched to the task need, and good formative evaluation. I will not dwell on these topics in this review, but they are worth reading to understand the direction of his research in the future.

 

References
Horn, R. E. (1989) Mapping Hypertext: Analysis, Linkage, and Display of Knowledge for the Next Generation of On-line Text and Graphics, Lexington MA, The Lexington Institute

Pask, G (1976) Conversation Theory, Amsterdam, Elsevier.

 

Acknowledgements
Thanks to Bill Verplank for useful comments on an earlier draft of this review.

 

Reviewer
Robert E. Horn is founder and Chairman of Information Mapping, Inc., an international training and documentation firm and author of the recent book, Mapping Hypertext. He has taught at on the graduate level at Columbia, Harvard, American and Sheffield (U.K.) Universities.

 

Information Mapping, Inc. 300 Third Avenue, Waltham, MA 02154.
Phone (617) 890-7003 (Master Card & Visa accepted.)

www.infomap.com

Information Mapping and InfoMap are trademarks of Information Mapping Inc, Waltham, MA.

Home