From the Society for Organizational Learning, Assessment for Learning Research Initiative, First Research Forum - January 14-16, 1998
In Three Parts: Part Three
The meeting concludes with a discussion about the process of business measurement. How do we know? Prof. Johnson takes the view that profit measurement is the wrong target and that the effort put into that is misdirected. The speakers seem to come to the conclusion that in a firm where the "music" is right, that the work produces real time feedback which is used to monitor the quality of the work.
(What follows are some segments of the conversation that followed Tom Johnson's evening presentations at the workshop)
Question (Lori Breslow) : I think there are some questions that are answered very well by measurement and there are some questions that aren't. In my view, the real problem has been in many fields in the social sciences, for example, we have tried to answer certain questions where measurement should not be used.
It worries me that you said you would throw out, what I think, an important tool. I think the real question is, what is it that we're trying to learn about and what methodology best helps us to learn about that?
Thomas Johnson : Well, I'm not advocating that we eliminate the descriptive nature of measurement. It's probably inevitable that we will use measurement to describe features of systems. It often is very useful, just as long as we don't let measurement drive actions. Once we say: "How can I take apart the pieces of what I've just measured and make what I've measured somehow better or different, or more or less?" That's where I draw the line.
Tim Savino : I would add that the question of why are we measuring may be also thought about along the lines of, "Who is asking and why are they asking?" There are some virtues to trying to measure in order to understand, because measurement and understanding can be intimately tied together. It's when the measurement and understanding are detached, that I worry. When we measure because we've been told to measure, or in order that we can boss somebody around with our measures, when we measure and we really don't understand what any of this is for, that I think we do harm.
Thomas Johnson : I agree, Tim, with everything you say about keeping measurement and understanding linked.
Tim Savino: To me, what I would hope to get out of this whole effort are steps toward better understanding, rather than an ability to say, "You are a learning organization, and you're not."
Peter Senge : Tom, I don't think you said Toyota doesn't measure anything. You did say something about how they don't use the measures.
Thomas Johnson : Yes, it's the way they don't use them that I find interesting, not the fact that they do not measure. Toyota measures; they just don't drive actions with quantitative targets.
Bill O'Brien : They don't use it to motivate action.
Thomas Johnson : Right, they don't use measures to drive decisions about how work should be done and what work should be done and so forth. Of course, they have an excellent accounting system. They invented what we call "target costing." But that's a descriptive measurement concept, really. It's anex-ante tool, employed before the work is even started. But once the work begins, cost targets play no role in influencing operational decisions. The things that guide the work come from a different level of abstraction than quantitative measures come from. Guiding the work are things that aren't measurable. Over time, they develop systems and patterns of behavior that are deeply ingrained in people. These are deep disciplines they have in order to know, "how is the work flowing?" Do we have a capability to detect normal from abnormal? These are the types of things they focus on, that everybody comes to know.
Bill Torbert : You're also making, it seems to me, an incredibly important point which some modern social scientists don't appreciate sufficiently at all. That concerns the distinction between ordinal measurement and interval measurement. We assume that we need to drive measurement to interval because we need to use the most sophisticated, quantitative statistics (at least what we consider the most sophisticated).
In fact, it's not clear that human psychology, except in extraordinary cases is capable of accurate interval measurement. As you say, what we measure in reality is "more or less." We can sometimes prioritize first, second and third. We can sometimes accurately distinguish nominally, sometimes. So, the more "primitive" forms of quantitative distinction, at least as we think about it in statistics, are actually the ones that are more important for the sorts of assessments we want to make in action.
But you do seem to take a very extreme position. You almost seem to come around to saying, "Let's get rid of that kind of measurement associated with the bottom line camp."
Peter Senge : What did you think about the point Bill made about throwing out the measurements associated with the "bottom line camp."
Thomas Johnson : I think profit's a descriptive measure that we can use, although I think it's a very crude and ambiguous one. I have particular concerns about profit which stem from the fact that I don't think we need it. We never had it, really, until the 20th century. Before the 20th century, for all practical purposes, what businesses had, coming out of double entry bookkeeping, was the balance sheet. The balance sheet told them how much they owned and how much they owed. Knowing how much they owned and how much they owed answered all the questions that bookkeeping was capable of answering.
The question the income statement purports to answer, "how well are we doing," was answered before the 20th century by real-time information in the hands of the guy who was out in the counting house, out in the plant, out with customers, or the guy who was getting news about how the ships were faring on long voyages. Those people did not use income statements to see how well they were doing. In this century, we somehow got caught up in the idea that we must have accounting measures of performance to answer this question. The more we relied on accounting abstractions to tell us how well we are doing, the less connected we were with the concrete, natural realities that generate revenue and profit.
Peter Senge : And then problems arose when that accounting performance information was used to guide the people doing the work.
Thomas Johnson : Yes. Moreover, the coming of income taxation amplified the importance of these accounting numbers and helped accelerate their use. But I think if we go back to the day when it all got started, we should have stopped and said, "Do we really need this?" "Do we really want this?" Is there, perhaps, more danger in it than help? Because we might change our minds. I would say, for example, if I could re-write all the laws, I'd go back and argue for having nothing more than standard double-entry bookkeeping with a balance sheet. Of course you also must keep track of cash flow, there's nothing wrong with that. If you do not do a good job of managing cash flow, you're out of business.
But that's not what income determination is about. I would say, get rid of income determination and the obsession with income numbers that drives so much of what accountants do today.
Bob Putnam: Could I go back for a moment? When you were saying that Toyota does not measure the work in order to drive behavior, you also said that they do attend very closely to how the work is being done, and they try to imbue everybody in being able tell the difference between when the work is flowing normally and when it is not? So I'm thinking, they are very good at assessing the work practices -- not with measurements, but assessing in the sense of knowing "Is it going well or is it going downhill?" They can build a pattern. And that's, arguably, something that makes them Toyota.
Thomas Johnson : Yes, I don't know what words to use for this. One way I've described this in what I'm writing now is to say that they have a very highly developed immune system. Through the disciplines they have imbued in their people, through the training over the years and the standardization of work and so forth, they know, as Arie de Geus put it, "what is us and what isn't us." Everybody knows this, in everything they do. It isn't through measurements that they know that. They know it in the same way your body's immune system "knows," which is probably not through measurement either. It knows it by doing it.
Bob Putnam : But it is getting information and assessing that information.
Thomas Johnson : Well, the information is the work, the work is the information. It isn't coming from anywhere outside the work.
Bob Putnam : But I'm saying that, if you want to use the analogy of your immune system, your immune system is in fact getting information from the foreign substances and assessing that.
Thomas Johnson : Well, it's talking to them. In a sense, the molecules in your body are talking to the molecules that shouldn't be there and they're having a conversation. That's the way the biologist Francisco Varela puts it. I don't really understand this area very well, but that's the way he talks about it: there's a conversation going on. In a sense, the information is in real time, its part of the "work." Either we kick you (the foreign element) out, we kill you, or you join us somehow and you help us.
Bob Putnam : But I think the critical point here is that it's (a feedback mechanism) in real time. The assessment isn't something that other people come and do on a different time cycle from the work itself, and then reintroduce their conclusions at a later time.
Thomas Johnson : Yes, it's part of the metabolic process.
Bill O'Brien : And it's done by the people who are doing the work.
Thomas Johnson : Exactly.
Bob Putnam : It sounds as if we're starting to develop some criteria for what we would want effective assessing might look like.
Bill Easterday : I was talking with somebody who was head of a car plant who said, "You know, when Toyota went into somewhere in the Southeast where they have a plant, they sent 300 managers from Japan to sort of shadow the people in the plant for a year, coaching them." My god, what an investment. But, if I then think about what those 300 Toyota coaches were doing is transferring knowledge of recognizing when the process and the practices are going right, then it starts to really make sense.
Thomas Johnson : Actually, if it was Georgetown, Kentucky, I believe they sent approximately 600 people for about 3 years.
Bill Easterday : That's just mind-boggling. But from the point of what I'm thinking now, in the context of the current discussion, it starts to make sense. If you can develop the capacity of people on-site to recognize patterns in the way you have been describing, if they can recognize what is going right and what is going wrong so they can make corrections, if they can transfer that sort of knowledge through a three-year process of face-to-face, one-on-one coaching.
Thomas Johnson : Working together?
Bill Easterday : Right. That's a daunting challenge, but it does point to the fact that there is real knowledge for assessing what's doing well and what's not.
Thomas Johnson : I think in the context of what Juanita Brown writes about it, it's a conversation. I mean, it's a conversation going on all the time that's making this work.
Peter Senge : Come back for a moment to Tim's question about who is asking the question, because it seems to be particularly appropriate. We are painting a picture of a type of learning going on within a group of people responsible for and active in doing something.
But then there's somebody else over here, not involved. They have the temerity to ask the question, "How's it going?" Now, somebody else wants to know.
There are usually two sorts of people in corporations who ask these questions: "management" and investors. Both are not directly involved in the value creating processes, they are on the outside. So it seems to me that we've come up with measures that may have been of some use as post-hoc, retrospective ways for people on the outside to know how things are doing, measures like profit.
How do we answer them? Do we answer them? Do we tell them to go away? Do we tell them, "Look, it's ultimately a matter of trust, period, end of story. If you don't understand the process and how people are learning in the process, you have no right to ask the question."
Thomas Johnson : When you raise the issue of profit, again, I think that many people say, "we've got to have profit because there's a third party out here, the stock analyst, shareholders, the Internal Revenue." I think from a practical standpoint, this issue that many people are troubled with here. What are we going to do about profit? That can be answered if we got rid of the internal revenue code. I mean, if we didn't pay income taxes, particularly corporations, who'd give a darn? I think most of the accounting department could then be eliminated. Remember that famous picture, I think it was on the cover of Fortune, of the Chrysler controller or CFO standing next to Chrysler's annual income tax return. He was about 6'2" and this return loomed way over his head. If the need to prepare such a report were to go away, a whole lot of this concern about how do we measure profits, I think would also disappear. Because most of the other people in the organization and in the finance community aren't really that concerned about it anyway. I think an important political action we could all take is to write our Congressmen and press them to stop taxing income.
Juanita Brown : I really want to honour the journey that Tom has been on, and his courage to continue that journey and to articulate his understandings.
Thomas Johnson : Thank you. You've been an important part of it.
Peter Senge : Thank you, Tom, and everyone, for a most remarkable evening.
See also http://www.ate.co.nz/owndata/measurement.html
See also http://www.ate.co.nz/owndata/johnson.html
Printed from http://www.ate.co.nz/owndata/discussion.html