Emergent Behavior and the End of Management - 1

The Wall Street Journal recently proclaimed the end of management as we know it:

Business guru Peter Drucker called management "the most important innovation of the 20th century." It was well-justified praise. Techniques for running large corporations, pioneered by men like Alfred Sloan of General Motors and refined at a bevy of elite business schools, helped fuel a century of unprecedented global prosperity.

But can this great 20th century innovation survive and thrive in the 21st? Evidence suggests: Probably not. "Modern" management is nearing its existential moment.

Modern management techniques - the organization chart, breaking tasks down into simple, repetitive steps, procedure manuals, job descriptions, and all the rest - had to be developed as businesses became too complex for any one man to fully understand.

When Adam Smith wrote about the invisible hand forcing businesses to benefit society as they tried to benefit themselves, most businesses were individual proprietorships such as the butcher, the baker, and the candlestick maker.  A tradesman might have a few apprentices and helpers, but the businesses were small enough for the boss to know what was going on in every corner of the business at all times.  In addition, the "master" understood all aspects of the entire process as well if not better than any of his employees.

The Necessity of Management

As the industrial revolution progressed, economies of scale and specialization became evident.  It cost a lot less per loom for a mill to have 100 looms, and even less if the mill had 1,000.  Whereas weavers who operated individual looms at home had to know every aspect of loom operation including how to repair their machines, it turned out to be economical for workers to specialize in tasks such as setup, operation, and different types of repair.

Organizing the labor force to keep 1,000 looms repaired, running, supplied with raw materials, keeping track of the skill levels, working hours, and pay due each employee required that the art form we've come to know as "management" be invented.  A mill manager generally did not know how to do anything involved with operating an actual loom; his specialty lay in coordinating those who did.

Specialization also meant that each worker had to know far less than the individual weaver but, collectively, was far more productive in terms of salable product per man-hour.  This rapidly bankrupted old-school individual weavers. Not surprisingly, they protested, often by burning textile mills to keep unskilled, lower-cost workers from taking "their" jobs.

The textile workers lost the battle and the Industrial Revolution came into full flower.  It wouldn't be possible to manufacture automobiles, refrigerators, microwave ovens, or any of the thousands of products without which modern life would be impossible without well-honed sales departments, factories, supply chains, bookkeeping and financial planning, and a host of other specializations completely unrelated to creating actual products but essential nevertheless.

The Achilles Heel of Management

The Journal points out a major flaw in management nirvana - bureaucracy:

Other management icons of recent decades earned their reputations by attacking entrenched corporate cultures, bypassing corporate hierarchies, undermining corporate structures, and otherwise using the tactics of revolution in a desperate effort to make the elephants dance. The best corporate managers have become, in a sense, enemies of the corporation.

The reasons for this are clear enough. Corporations are bureaucracies and managers are bureaucrats. Their fundamental tendency is toward self-perpetuation. They are, almost by definition, resistant to change. They were designed and tasked, not with reinforcing market forces, but with supplanting and even resisting the market.  [emphasis added]

What does a large business do when faced by new competition?  Do businesses see competition as a sign that they need to connect better with customers and improve their products?  Or do they imitate the protesting textile workers and whistle up their lobbyists and lawyers to attempt to have the competition declared illegal?

To give corporate bureaucrats their due, most new ideas will not work.  When an organization is optimized to make, say, mainframe computers as IBM was, the mainframe bureaucracy sees attempts to do something unknown such as making PCs as a waste of time and resources which should be better spent making better mainframes.  Most of the time, they're completely right.

When they're wrong, however, the results of ignoring the new disruptive market force can be disastrous.  GM, Ford, and Chrysler first attempted to outlaw Japanese cars, then they failed to improve their products to compete and have all had near-death experiences.  IBM would have been gone without a genius leader who completely redirected the firm into consulting and brought it back to profitability.

RCA is a somewhat more poignant example.  I used to deliver executive seminars on artificial intelligence through Frost and Sullivan.  They booked me at RCA's Sarnoff labs where I was told this story:

Back when RCA had a lock on the tube radio market, their scientists developed a prototype transistor radio.  Marketing, fearing that tube radio sales would be damaged, wouldn't let them put it in production.

A delegation from Sony came through, saw the prototype, memorized the schematic, and the rest is history.  The RCA marketing department was correct - transistor radios obliterated the tube radio market, just as they had said.  RCA is no more, however, because it allowed the Japanese to replace its outdated products instead of doing it themselves.

The RCA marketing bureaucrats were neither evil nor stupid, but they realized that not only would transistor radios destroy the tube radio market, the new kind of radio would devalue their expertise in marketing tube radios.

Although they were both called radios, the two technologies were utterly different in effect.  The tube radio was plugged into the wall so you could listen at home.  The battery-powered transistor radio you could take to the beach.  Not only would the new technology require a complete rebuild of the radio factory, it would require a completely new marketing strategy, different sales channels, and a new marketing staff.

In torpedoing the new transistorized radio, the RCA marketing staff almost certainly saved their own jobs.  RCA didn't die immediately; the top-level marketing executives made it through to retirement although pensions were cut in half when RCA went under.

For all the other employees, though, the price of their shortsighted selfishness was vast.  It takes an unusual sort of person to sacrifice his own job to save his company particularly when it's not clear that the new product will save the firm; RCA's marketers did exactly what almost anyone else in their positions would have done.

How Ideas Die

The classic corporate organization chart militates against trying anything new.  If someone proposes a new idea to their boss, the boss can say "No" and that generally ends it.

The boss can't say "Yes" as effectively and finally, of course.  All he can say is, "Yes, but I'll have to check with my boss."  All the would-be entrepreneur can get is "Yes, but," "Yes, but," "Yes, but" all the way to the very top.  Oftentimes even the CEO can't say "Yes" and make it stick; vested interests elsewhere in the company will kill the project by degrees.

Suppose that the probability of a "No" at each level is only 50%. There's no way a troll six or eight levels below the top can get a "Yes."  50% "No" per level times eight levels means that one idea in 256 might get an ultimate "Yes."

In fact, rejection figures are closer to 90% per level which means that the probability of a final "Yes" is infinitesimal.

That's why it's so difficult for a major corporation to change course - the staff at all levels see all new ideas as threats to their current power, status, and the value of their skills.

The unarguable fact that most new ideas fail helps the staff justify their efforts to protect their turf.  After all, how could AT&T executives who had a century of experience providing high-quality, reliable voice telephony imagine that customers would put up with lousy sound quality, constantly dropping calls, and incompetent customer service from cell phone companies?

The bottom line is that the corporate bureaucracy is statistically correct to squash most new ideas, but when a disruptive idea that actually works comes along, they can be put out of business very rapidly.  Such disruptions used to be relatively rare events, but they've become far more common as billions of Chinese and Indian would-be entrepreneurs got government permission to join the world economy and the Internet speeds the spread of information and new ideas.

As the Journal put it, "Decades-old institutions like Lehman Brothers and Bear Stearns now can disappear overnight, while new ones like Google and Twitter can spring up from nowhere."

What's the Alternative?

We needed management to enable the industrial age.  To the extent that we continue to need large, complex factories to product complex products such as automobiles, we'll continue to need classical management.  Even in the auto business, however, managements are becoming more and more flexible and less and less hierarchical as new ways to organize such enterprises are tried out.

For example, a Ford plant in Brazil is experimenting with locating supplier plants on the same site as the Japanese do.  This cuts transportation costs and quality issues surface faster but it requires a great deal of trust.  It's one thing to stop buying from a supplier who's located in another city, it's quite another matter to stop buying from a supplier who has a facility on your own site and whose employees eat lunch with your employees every day.

Issues such as managing complex supply chains and the engineering needed for complex products mean that we'll need classic management for some time yet, but how can large companies become a bit more nimble without giving up the efficiencies of classical management practice?

The next article in this series discusses a new organizational form where a number of autonomous agents organize themselves to achieve a mutual goal.  It's called "emergent behavior."