A More Efficient Model For Problem-Solving Across Hierarchies / by Daniel Ehrman

We work in organizations where different people have different specialties in different fields of knowledge, all of which we depend on in one way or another. We have some worker in Site A who's an ARM expert, another one in Site B who knows DDR, and a third one in Site C who understands verification IP. All of these people are solving their own problems independently, despite the fact that most problems consist of a variety of contributing knowledge bases; the worker pushes on through the problem, struggling to understand what he doesn't already know—but what someone else does.

We can think of this situation like a Venn diagram with each person having his own circle of knowledge or expertise. Each person approaches his problem by sort of pecking in the dark all around his circle, maybe expanding his search a bit, hoping that he eventually hits the slice of area intersecting with someone else's domain. A real-world example may be, after asking around the office and bouncing along a chain of e-mails for a few days, finally finding the person who can fill the knowledge gap—and even then, often, only partially.

Perhaps more concretely though, it's best to think of the situation as it fits in with the typical hierarchy:

Hierarchy.png

This kind of a structure will be as present in a computer engineer's mind as it is in that of a human resources department. And as any computer engineer could tell you: those "leaf nodes" at the bottom of the tree—they're isolated from each other. In fact, it's inherent in this structure that the only way of getting from one leaf to the other is to go up the tree through each node's "parent."

So what's the point? The fact is passing information between leaves (i.e. lower-level employees) is remarkably easy these days with e-mail, web conferencing, and other collaboration tools. The key insight though is to recognize that the leaves only see what they've been given, and they will only communicate with each other when asked. So although in reality they may have direct connections (phone, e-mail, etc.) they remain isolated by what they're told to work on.

Taking a step back, let's think about how a task makes it's way to the individual problem solvers at the bottom of the tree. High-level goals are constructed by upper management; criteria are defined; and tasks designed to meet those criteria are assigned to workers at the next lowest level, with increasing amounts of detail at each level down.

But—the point at which a problem is assigned to a specific node at the next lowest level, an important (and potentially costly) decision has been made: the other nodes at that level, and all of the workers beneath them, have just been pruned from solving that problem. And at that moment, the decision-maker must operate with only the limited information he has been provided by the single level beneath him.

As it turns out, this is a classic problem in artificial intelligence: in the searching of decision trees, such as in the game of chess, often there are far too many possibilities to consider in a reasonable amount of time. The solution is to develop a heuristic, search only a few levels below oneself, and make an educated guess about what you think is probably the best move.

This is effectively how the business world is working now, and has been for a long time—managers making big decisions with highly filtered information.

But with the technology available to us these days, there is no reason to keep proceeding with this outdated approach. Think about how you find an answer to a question when you're searching with Google. Do you browse through a hierarchical list of categories, at each step of the way considering all of the links and choosing the one which poses the highest likelihood of containing your answer? No. That model of search has been dead for years now because people understand the power of letting the algorithms choose your results for you (based on their unbeatable knowledge of millions of available options).

So where is the Google of problem-solving in today's businesses? Where are the "page rank" algorithms for telling managers the best resources to assign a task? Imagine indexing everything related to a problem in a centralized database: products affected, people involved in the solution, who knew a lot about it (and who didn't), tools used, etc. And imagine that every time a person contributes to a solution, his information is linked to that problem, effectively creating a "portolio" of problems that worker has under his belt. Finally, we can pull in live data like individuals' schedules and project timelines to ensure that resources are always properly allocated.

Over time, we have a map of our resources embedded in this database, and we can use "big data" algorithms to intelligently assign workers to a task based on their comparative advantage.

The final result is something like matrix management but with even less barriers between teams. The new style dramatically crushes information silos and makes tribal-style management difficult to exist at all. But here it is important to be wary of a potential motivational pitfall that may result from these new highly cross-functional resources: in fact, the flattening effect that occurs when all resources become available to all possible problems may actually induce competitive issues at managerial levels. A manager is responsible for the work of his team, and thus if his people are being pulled into other teams' work, it will be difficult for the manager to find reason to support such "outside" activities.

Therefore, if such a system were implemented, it may be most effective to apply the traditional resource-pruning at a moderately high level while still leveraging the power of the algorithm's decision-making based on its in-depth knowledge of all available resources. Even still, a front end for the problem-solvers database could be provided to the lower-level employees to give them a means to get answers more quickly. In this case, employees would have the power to solve their problems efficiently, while the managers would still have the power to decide how those employees should spend their time.