There's despair afoot on the campus of Blogistan Polytechnic Institute. People are still losing jobs at an alarming rate and our financial system remains in disarray, yet President Obama and the Congress are dithering on how to fix things. Universal access to health care? Maybe we'll get it and maybe we won't, someday. The wars in Iraq and Afghanistan? Again, maybe we'll end them and maybe we won't, someday. Meet the new boss, same as the old boss? It can certainly seem that way at times. There are real problems that demand real solutions, so just solve them already, right?
But what is a "best" solution, or even a "good" solution? In complex systems with so many unknown elements, some because we haven't measured them, others because they're in constant flux, and so many interrelated causes and effects, how can we know whether any given solution is "best," or even "good?" And assuming we can know - because the alternative is to throw up our hands and give up - why don't our leaders stop dithering, pick the best solution they can get done, and just do it already?
More below the fold....
What is a "best" solution?
We're always searching for "best" solutions, though we more often settle for what we think or hope are at least "good" solutions. But how do we search for them, and how do we know when we've found one? Our personal method is often intuitive: invisible reasoning processes the facts and rules we know or assume to be true and bubbles out a sense of "Yes, that's good." But in a democracy, political questions are supposed to be resolved by debate, and that requires visible reasoning. We have to state the facts and rules we know or assume to be true and work forward. Or, commonly, we decide what solution "feels" right, work backward to the facts and rules that would justify that solution, then present them as if we knew them to be true.
Get 435 representatives, 100 senators, one president, and countless other voices all doing that at once, and you get a dizzying assortment of "facts" and "rules" that are hopelessly contradictory, most of them unverified, and many of them unverifiable. If it were nothing more than that, it would be government by trial-and-error, where the trials are always hugely expensive and the errors often disastrous. And cynics would tell you that's exactly what we have.
I think they're wrong, and the way they're wrong goes back to that first clause: "We're always searching for 'best' solutions." Because it turns out there's a field that studies how we search for best solutions, and how we estimate that we've found one.
That field of study is called heuristics. The word itself comes from the Greek Eureka ("I have found it!"), made famous by the story of Archimedes running naked through the streets after discovering in his bathtub how to measure the volume of an irregular object. You probably remember the story from school. (You may also remember thinking Mrs. Archimedes must have been right behind him yelling "That's great honey, but put some clothes on!" Or maybe that was just me.)
Today we'll explore the second half of heuristics: how we estimate that we've found a "best" or at least "good" solution. I think it reveals a lot about why our democratic process can seem so dysfunctional and still function.
"I have an idea for a book."
As a novelist, I must have heard that sentence a thousand times. It seems to be the second most common response when I tell someone I'm a novelist. (The most common being "I could never write a book.") It's also a sentence I say to myself often enough, at least by implication, when I think I've found an idea that could be a novel. Key phrase: "when I think I've found."
Because it turns out that's a very inexact process. There are a lot of things a novel needs to be, if you write novels for a living. It has to be 55,000 to 100,000+ words, depending on the type of story and audience and publisher expectations. And it has to be salable; if no publisher will buy it, then however beautiful it might be to me personally, it's not going to help pay the bills. Yes, I am a commercial hack. So were Shakespeare, Dickens, and Tolstoy. Sorry.
But back to that 55,000 to 100,000+ words bit. That means the idea has to have enough "there" there to play out over that length. Too little and the story will read "slow." Too much and it will seem "dense" or "contrived," the latter because there wasn't room to fully develop the character motivations and vivid moments that allow the reader to set aside his/her awareness of contrivance, or what Samuel Taylor Coleridge called a "willing suspension of disbelief."
Having authored and co-authored (with Herself) a lot of books, I've developed at least some intuitive sense of whether an idea has enough "there" there. I call it "a story that will tell." It's not formulaic and it's very imprecise. It's a constellation of "things that tend to work out well." It's a set of heuristics.
And you probably have the same in your profession. They probably aren't strictly formulaic, and most of them are very imprecise. But you balance one against another, explicitly or intuitively, and develop some sense of "This is a good solution." It's your set of heuristics. Some of them may be unique to your profession, but usually they're not so much unique ideas as they are unique applications of common ideas.
Some common heuristics
It turns out there are some common heuristics for real life solutions to real life problems. This is not a complete list, and in fact it's very selective for a reason we'll explore below:
- Is it principled? Each of us has a sense of what is moral and ethical, and if we're faithful to those principles, that sometimes means we can't adopt a solution that would fit our other heuristics, because we judge that it wouldn't be the right thing to do.
- Is it practical? On the other hand, a perfectly principled solution might be a non-starter. We can't convince enough other people to get it implemented, or its successful implementation would require resources we don't have, or for people and/or the universe to behave in ways we believe are unrealistic.
- Is it simple? We generally prefer simple solutions over more complex solutions, because we're better at doing simple things than we are at doing complex things. That's especially true when the solution requires the cooperative action of a lot of people. The more complex the solution, the more likely that enough people will make mistakes that the solution won't work.
- Is it flexible? On the other hand, the universe isn't simple and simple solutions often fail to account for that complexity. The more we try to tailor a solution to be flexible enough to deal with unexpected events, the more complex the solution will become.
- Is it efficient? We have limited resources, and we don't like to waste them if we can avoid it. A good solutions uses as few resources as possible, preserving unused resources for other problems.
- Is it robust? On the other hand, all solutions fail from time to time, because we run into unexpected events, or simply due to human error. If a solution is too efficient, it has too little margin for failures, and a little failure will cascade into a complete failure. A robust solution will have failure-alert and failure-correction built into it, and those require "extra" resources in the form of monitoring, maintenance, and/or redundancy.
- Is it familiar? We tend to do best what we already know and have done before. New solutions require us to use strategies that we may not understand well enough to implement effectively. A new solution may be better "in the long run," but if we fail too badly during the learning curve, we may not get to the "long run." And new solutions carry an inherent bias in favor of those who learn faster, over those who need longer to learn a new system.
- Is it novel? On the other hand, this probably isn't the first time we've encountered this problem. Indeed more often than not, it's a problem we've had for awhile and haven't been able to solve. Is this solution different from what failed before, or are we repeating the same mistakes and hoping for a better outcome?
Heuristics often come in complementary pairs.
That is admittedly a limited set, and I chose that set for a reason. Each is part of a complementary pair. The odd-numbered heuristics offer one characteristic of a good solution, but each even-numbered one begins with "On the other hand...." It turns out that's a very common (though not universal) property of heuristics. When you try to maximize one, you usually sacrifice another. And you can't always "split the difference," as that assumes the two carry equal consequences.
And in democratic government, "therein lies the rub." There is often a set of demonstrably bad solutions, but rarely a demonstrably best solution. We can argue a given solution is "good" as measured against one set of heuristics, but someone else can argue that solution is weak as measured against a different set of heuristics. We might agree that all of those heuristics are important, but disagree as to which are most important for a given problem. We muddle through as best we can, balancing principle against practicality, simplicity against flexibility, efficiency against robustness, familiarity against novelty, and so on.
And always there's someone ready to say "That's not good enough."
Happy Thursday!