Skip to main content

Last night I was privileged to hear a lecture by Dr. Scott Page, who, if you're not familiar with him, is a professor of complex systems, political science and economics at the University of Michigan. His speech was all about how diversity is critical to the success of a business, indeed, he wrote a book about it, presenting a mathematical theorem arguing that, especially in the field of making predictions, the more diverse a group, the less crowd error there is.  This got me thinking--there's a lesson for the GOP (and perhaps, the Dems and bipartisanship as a whole) in the making here.  

So, join me as I attempt to recreate his argument and tune it toward politics. In doing this, I also hope to prove the importance of the netroots, why Obama was brilliant in picking Leon Panetta, the importance of a variety of people entering politics rather than just the most privileged (i.e., why a career politician like Jim Tedisco might not be as good a choice as a political newbie like Scott Murphy), and the importance of an active, constructive bipartisan effort.  Oh, and how we might work to fix the financial crisis.  High hopes, huh?

First of all, let me say that I am not a math person in the slightest.  As such, I will to make this as readable as possible, and any errors I present here are mine, not Dr. Page's.  The book, incidentally, is The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies.  I've just begun it, so I'm not an expert yet.

Let's begin with the basic ideas behind Dr. Page's argument.  We will begin with an example, quoted from Dr. Page's book (pg. 179, in case you're interested):

In 1906, Francis Galton analyzed predictions of the weight of a young steer by attendees of the West of England Fat Stock and Poultry Exhibition.  The average (mean) of 787 contestants' estimates of the weight was 1,197 pounds.  The actual weight of the steer was 1,198 pounds.

Now, Page isn't arguing that all you have to do to be right is to get 787 people in a room and have them guess something.  His argument, in terms of making accurate predictions, is that having a diverse crowd making your predictions is key.  If everyone at the Exhibition was a stablehand, for instance, the prediction might not have been as good.  Instead, you have stablehands and farm owners and ranchers and poultry farmers and everyone else who came, each with their own perspective.  Even though everyone isn't an expert on weighing steers, the average of all answers, inaccurate and accurate alike, yielded a dead-on prediction.  Page expresses his finding as "The Diversity Prediction Theorem":

Collective Error=Average Individual Error-Prediction Diversity

As I understand it, what this means is that each individual person is bound to be wrong by some margin, large or small.  How wrong they are is going to be different from person to person.  It is the diversity of the group that offsets this.  Every person makes their predictions using what they know, meaning that the collective (that is, the average) prediction is going to benefit from everyone's individual knowledge and experience, offset those who bring the wrong information to the table, and come up with a prediction that is closer to the correct answer than everyone's individual guess could be.

Still with me? Yes? No? Here's another example (I'll try to be quick--feel free to skip this if you follow me.  I'll let you know when it's safe to come out.)
Page doesn't like online IQ tests.  As he said last night, they tend to place his IQ in the thin band between 80 and 225.  One of the questions it mentions he finds odd is the "fill in the number sequence problem".  For example, we have:

1 4 9 16 __ 36

This sequence even I, no master of math, can figure out.  The square of 1 is 1.  The square of 2 is 4, the square of 3 is 9, and so on, making the missing number 25.  Here's another.

1 2 3 5 __ 13

This is a little harder, but some here might do the subtraction necessary to realize that this is part of the Fibonacci Sequence.  Subtract 13 from 8, you get 5, subtract 3 from 5, you get 2.  Thus, the missing number is 8.  

The third example is where it gets interesting:

1 2 6 __ 1806

I didn't know what to do with this.  Nobody at the dinner last night knew what to do with this.  Page told us that he presented this problem at a dinner for the International Monetary Fund, and none of them knew what to do with this.  However, Page says that when he presented this to an inner-city school in New York, 4 of the kids knew the answer to this!

The answer, especially for those of you who love Douglas Adams, is 42.  Why?

Well, in the two sequences so far, we've done multiplication (a number times itself), we've done subtraction.  Why not both?

Each number differs from the number after it by an amount equal to its square 1=2-1^2, and 2=6-2^2. This idea seems cute, but it doesn't seem as though it will get us to 1806.  And yet, it does.  Using this rule, the next number would be 42, 6=42-6^2, and the number after 42 would be, (guess what) 1806.

Again, the IMF couldn't figure this out, but four different high school math students could.  They've been doing plenty of multiplication and subtraction.  No idea why the IMF couldn't.  I guess they're a little busy.

OK, MATH-HATERS! You can come out now! It's safe.

Here's MiscellanyMan's Conclusion #1: We need more than career politicians in Washington, or in any government, state, city, county, etc.  A few aren't bad, as their perspective and ideas add as much diversity as somebody else.  But we need more than just them.  Thus, you can make the argument for NY-20, that there are plenty of career politicians representing us in Congress, both Democrats and Republicans.  Is one more (Tedisco) really going to help us, or should we look to add another useful perspective (Murphy)?

More from Page now.  His big message at the dinner last night was that ability and diversity are equally important.  Look at the example above.  Presumably, the members of the IMF are better at most complex math problems than high schoolers.  But when one is stumped, they all are.  They all have similar enough perspectives that they couldn't figure it out.  The high schoolers in that room were only there for three reasons: Where they live (that's why they're at that school), what their age is (why their in that grade), and which hour math class they're in.  Otherwise, they could be black, hispanic, white, straight, gay, lower class, middle class, who knows.  They have a diverse array of perspectives, and were able to figure it out.  

Why should Congress be any different?

MiscellanyMan's Conclusion #2: If the GOP keeps forcing lockstep party discipline, they will remain sunk.  Now, most of us here already knew that, but isn't it nice to know that there's some math behind it? (If you want more, go grab Page's book--there's a nice big chapter titled "The Empirical Evidence".  I haven't gotten through it yet, but I'm reading my way toward it.

Another point Page made last night is that a variety of ideas is crucial to problem solving.  He made the point with a number of visuals I can't reproduce as well in text form, but I shall at least try.

Let's say you have some people tasked with a difficult problem.  If it was an easy problem, after all, anyone could do it, and you wouldn't need experts in that field.  So we have a difficult problem, and we have some experts.

Now, lets say we take the five best people in this field.  For purposes of our example, they each have three skills to bring to bear on this issue.  I'll label them using letters of the alphabet.  The closer that skill is to A, the more applicable and useful that skill is to the field in question.  Let's meet the experts.

Expert 1: ABC
Expert 2: ACD
Expert 3: ABD
Expert 4: BDE
Expert 5: ACE

Now, we also have a pack of random people from various other fields.  Maybe some are related to our field of choice, but not all are.  Let's meet our Random People:

Random 1: AEH
Random 2: BFG
Random 3: ADR
Random 4: BCJ
Random 5: IL

Now, these five experts are proficient in the main skills that make up this field.  If you had to pick one person to do everything, you'd probably pick Expert 1, as he's got skills A, B, and C, which are the three best.  But, if you wanted a committee to try to tackle this problem, would you really get more information if you repeated the same skills over and over as we have in our panel of experts? I mean, experts 1 and 4 have all the skills experts 2, 3, and 5 have.  Do we need them too? Maybe a little redundancy doesn't hurt, but do we need all 5?  Probably not.  

But look at the random people.  They're not going to be any better! Some of them don't even have three skills (and yes, Page did use IL in his example, and yes, he did make a jab at the state that produced Blago and Burris when he pointed this out).  But what if, in a case such as the horrific implosion of the financial markets, skill I or skill G brings us the best answer to the problem?  Think of all the inventions out there that were discovered purely on accident.  Page was asked during the quick Q&A that evening what he thought about the diversity of Obama's appointments, and Page praised him for picking an economic team that had economists, economic historians, and experts in other areas of economics.  He also said that you could do more.

MiscellanyMan's Thought Experiment on it's way to a Conclusion #3: Let's pretend that we need to reform the CIA.  Now, the CIA is a conglomeration of all sorts of talents.  There are analysts, agents, commanding officers bureaucrats, etc.  But, they all have some similarities.  They often compete with the NSA, FBI, foreign intelligence agencies, etc., and maybe don't always work nicely with such groups.  Some, though not all, might decide that torture is a great idea.  Then they might decide it would be really nice if they didn't get into too much trouble for thinking as they did about torture.  Now, we could appoint a CIA chair who has been part of the problem all along.  We could appoint a person who maybe has been out of the intelligence community for a while, but worked in it back in his youth.  Both might approach the problem in the same way, though, even if their hands weren't personally crafting the torture policy.  It's skills ABC vs. ABD vs. BDE all over again.

Or, we appoint, say, Leon Panetta.  The CIA is full of those basic CIA skills, after all.  But maybe, when it comes to picking a leader, we don't, to reuse the phrase, want more of the same.  We shouldn't remove everyone from the agency (really, we shouldn't), but that doesn't mean that we can't have someone on top who can take the info he's given and look at it in an entirely different light.

MiscellanyMan's Conclusion Lightning Round!
Conclusion #4: Maybe it wouldn't hurt to get a few more voices and perspectives outside of the world of finance as we move forwards.  Jon Stewart, for instance. Lord knows there seems to be a lot his fake-news skills bring to the world of real news.
Conclusion #5: This is why the netroots is so critical.  By getting everyone, and I mean everyone involved, we can create a group so diverse, so varied in perspectives, that our collective error will be much much smaller than the errors we no doubt make on our own.  We might even, say, win lots of elections in '06 and '08.
Conclusion #6: Re-read Conclusion #2.  Consider what the GOP has done since '08, on the stimulus, on the omnibus, on everything.  Re-read your favorite quotes from Limbaugh, Boehner, McConnell, Steele, Cantor, etc.  Of these, Steele is the one who probably fits the least.  How was his job security doing again?
Conclusion #7: We may complain about the DLC, Blue Dogs, Lieberman, etc.  I know I don't always agree with them.  But, I'm going to say this: Maybe it's good they're there.  Not just to fill seats and keep our majority, either.  

Now, I'm sure some of you won't agree with #7.  You might not also agree with the incoming conclusion #8, that bipartisanship is really important.  REALLY important.  

Before you send comments my way, however, let me clarify one thing I didn't tell you in the example above with the experts vs. the random people.

There was a proviso Dr. Page added concerning the experts and the random people, and that is that everyone in both groups

must be smart.

Don't start deriding the intelligence of your favorite target yet.  Here's my point, and, in turn, what I think Page means.

In order to be a part of the problem-solving process, you have to be willing to work to solve the problem.  If you're not, it doesn't matter if you have skills ABC or XYZ.  Now, the GOP wants to solve the problem of their party's identity, they're just clueless how to do so.  Whatever skill/letter of the alphabet they need, nobody in leadership has it.  However, I'm pretty sure that they could solve their problem by pursuing an active, constructive effort toward bipartisanship.  Now, in the short run, they don't stand to gain much, or so it seems.  Many times, during the discussion of the stimulus, I read something like "The GOP doesn't stand to gain anything from helping pass the stimulus--Obama and the Dems will just get all the credit, so of course they're going to oppose it".  This isn't necessarily wrong--I myself have said it a number of times while debating politics with people.  But what the GOP could have done is say No for reasons other than for the sake of opposition.  They could turn off party discipline, let the members vote their minds, call the Whips away from looking over Cao's shoulders, and be constructive.  

I don't think they'd get any credit for doing this.  Not a lot, at least.  And I don't think they should do it just to make the lives of Dems in government easier.  But if they did, the Republican's image would, I believe improve.  Maybe not this month, or this year, but at some point, the public might say "You know, those Republicans, they've been all right" and some people might start voting for them again.  The GOP would need to reconsider their policy priorities as they do this, of course, but anything is better than the shouting and redundancy that's been going on for the last several years.  

So, here's MiscellanyMan's Conclusion #8: If we want our government to have the lowest collective error, we need to include everyone.  The more diverse our government, the better.  Dems, Republicans, Independents.  Maybe get some other parties in the mix?  If Page is right, it will mean better governance.  And that's what we're all here for, right?

Now, all we have to do is set up the big spinning hypno-disks and convince the rest of the country that this is what we need.  

MiscellanyMan's Conclusion #9: Wow, that's easier said than done.

Originally posted to MiscellanyMan on Fri Mar 13, 2009 at 03:13 PM PDT.

EMAIL TO A FRIEND X
Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags

?

More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

  •  Tips for Diversity (7+ / 0-)

    Feel free to offer requests for clarification as well.  I know this is all a lot to handle.

  •  Don't you think this is overgeneralizing a bit? (3+ / 0-)
    Recommended by:
    HiBob, linkage, MiscellanyMan

    Politics isn't just about people's ability to apply expertise in coming to solutions that are empirically obvious. That was the messy downside of the Enlightenment - politics isn't just a matter of discovering the general good via reason, and it's impossible to try to get that kind of rational consensus. What you end up with is Robespierre and the Jacobins deciding that what they believed in was rational, and that people who disagreed with them were either fools, insane, corrupt, or in league with the enemies of France.

    People will disagree about things what the question is, what the solutions are, how you arrive at getting the solutions, which questions are more important than others, whether solutions or methods of arriving at them are more important. And they'll disagree, not moving towards some collective error, but all over the place.

    Why? Because they have different interests and ideologies and environments and intellectual backgrounds, etc. Political parties emerge out of common constellations of these disagreements, and allow for people to exercise their beliefs and thoughts and theories, if they can get a majority of voters to go along with them. You can't just expand these constellations indefinitely without losing agreement on an agenda for action.

    •  Oh, most likely. (0+ / 0-)

      What I'm arguing is that, to use your example, that Robespierre plus those who were rational and thought he wasn't plus any other points of view around there at the time.

      As for everything else you've said, I agree entirely.  But in politics, we strive to find politicians representative of their districts (and, arguably, don't succeed most of the time).  Every American has their differences, and they vote how they do in part because of those differences.  The elected officials we do have are also different from their colleagues (some more than others).  But we can't have 435 parties, just as we shouldn't have one party.  Politics is all about coalition-building.  But if you and I are in a party, and we have to come up with a policy position, even if we agree, we might do it for different reasons.  That, and people naturally categorize themselves.  Page actually had a lot to say on how humans categorize themselves and each other.  I can expound on that if you're interested.

      •  Ok (0+ / 0-)

        But either there's a boundary to party coalitions based on certain core principles, or the parties don't really exist, and can't really function, because not enough people are willing to work together on any given issue.

        •  Well, the difference in party coalitions... (0+ / 0-)

          ...is probably a function of getting elected.  Political parties are popular for the same reason as franchises--slap a name on something, and you have a general idea of what you're getting.  When trying to get elected, it's much easier to say "I'm a Democrat" or "I'm a Republican" or even "I'm a Green" than to say "I'm this on abortion, this on free trade, this on gay rights, this on the stimulus, etc."

          Now, I have examined elections in other countries, where there are dozens of parties that all have to get along.  In many cases, when the partisanship gets fierce, things don't get done.  But the same could be said of the Congress during Dubya, when we had nothing but deadlock, and the phrase "Do-Nothing Congress" came back into fashion.

          All of Page's arguments focus mainly on groups willing to work together, which is probably why he's focusing on diversity in schools and businesses and why I'm the one taking his arguments and applying them to politics, which are a much more confrontational enterprise.

  •  Nice work (3+ / 0-)

    as a math geek I loved it!  This also illustrates why we have a financial crisis, people who were bad at math. They didn't realize having large numbers of very closely related items (lots of sub-prime mortgages) was diversification it was only repetition. Truly spreading risk involves combining many dissimilar classes of assets. In politics spreading the "risk" of coming up with "bad ideas" requires many different "classes" of people, in large enough numbers to give them all "weighting."

    -1.63/ -1.49 "Speaking truth to power"

    by dopper0189 on Fri Mar 13, 2009 at 03:36:23 PM PDT

  •  Easier way to show solution to IMF stumpper: (2+ / 0-)
    Recommended by:
    TheCid, MiscellanyMan

    1^2 + 1 = 2
    2^2 + 2 = 6
    6^2 + 6 = 42
    42^2 + 42 = 1806

    I just upgraded internet speed. Now I can be late to the best diaries, faster.

    by mississippi boatrat on Fri Mar 13, 2009 at 03:47:52 PM PDT

  •  I don't get this (2+ / 0-)
    Recommended by:
    HiBob, MiscellanyMan

    You said

    1. In order to be a part of the problem-solving process, you have to be willing to work to solve the problem.  If you're not, it doesn't matter if you have skills ABC or XYZ.

    and then you conclude

    1. If we want our government to have the lowest collective error, we need to include everyone.  The more diverse our government, the better.  Dems, Republicans, Independents.  Maybe get some other parties in the mix?  If Page is right, it will mean better governance.  And that's what we're all here for, right?

    I don't see how 1 and 2 can go together given the nature of the Republican party (not individual).

    What you are missing is that the theory of more is better only works if the elements are at least neutral. The Republicans are obstructionists.

    •  it also leaves out the element of time (1+ / 0-)
      Recommended by:
      MiscellanyMan

      Let's reimagine Galton's experiment, but this time  all 787 people have to form committees, subcommittees, a charter, meeting schedules, platforms, etc, in order to come up with the way upon which they will all agree upon one answer. Furthermore, at least 10% of the participants have the ability to stall or veto the process.
      In politics, and most other things with no empirical answer,  for each voice you add to a decision making process, you add to the time. For each voice that has a veto over the process, make that an exponential increase.

      •  I'll agree with this (0+ / 0-)

        HiBob and iBlue both make good points--points that Page didn't address in his talk, and hasn't addressed in his book (again, he focuses more on much smaller bureaucracies that politics), and I think this may, in part, go to show how the Republican's current tactics will come back to haunt them.  By not allowing for a diversity of thought in their ranks, they prevent the progress they might eventually come to get credit for, even if their floundering party isn't rescued immediately by doing so. As of right now, iBlue makes an excellent point--Republicans, especially in the House, seem dead-set against anything but the party line.  By reducing that diversity to 0, their collective error is...well...just look at it. It's comical.

        My argument is not that the Republicans are doing this right now--but this amount of efficacy is what they might be able to achieve if they changed their ways.  I doubt it will happen, and as such we might be able to make the electoral argument that, in times when one side has left the bargaining table, it is in our interest to elect representatives who would take a seat at the table and not try to run it into the ground.  What does this most likely mean? More Democrats--but it may also mean that we will come to see more of a contrast between the various ideological factions within the Democratic party as well.  After all, when the Republicans check out, that's what we're left with in terms of ideological diversity. Did that make sense?

        As for HiBob (Hi Bob!) I think you make an excellent point.  Page only talks about the accuracy of decision-making, not the speed of those decisions. Though I wonder, as I try to fit Page's views into the political sphere, whether we might be able to cast more light on the various obstructionist measures that happen--not as a way of helping diversity, but simply as a way of shoving out those who don't want problems solved and replacing them with those who do (i.e., lets have more Gilchrists and Crists than Boehners and Sanfords.)

  •  why we need more than 2 parties (2+ / 0-)
    Recommended by:
    linkage, MiscellanyMan

    this is why i think our 2 party system is bad. we need 6 parties. something like this: neocons, libertarians, republicans, democrats, greens, socialists. the more diversity the better.

  •  I approve of diversity (3+ / 0-)
    Recommended by:
    itsbenj, HiBob, MiscellanyMan

    and I favor Instant Runoff Voting and Public Campaign Financing to get more voices into our political system.

    But I hesitate to accept "bipartisanship".

    When half the crowd is intentionally overestimating the weight of the steer because they're each getting a small bribe from the guy who wants to sell it... well you, then you have the Republican Party at work.

    BushCheney Inc. - They lied to me, they lied to you, they lied to our troops.

    by jjohnjj on Fri Mar 13, 2009 at 04:46:14 PM PDT

  •  lost me somewhat at (0+ / 0-)

    the end there, the Blue Dogs are not arguing a serious position, and I think that is really quite obvious to anyone looking at the situation. they offer no alternative, they're just pure opportunists. and they're allowed to be with no political price. a real shame. and the 'bipartisanship', and its' over-valuing. can't really abide that. there are legislative tools that could be put into place, ugly as it may be, to bypass the Republicans in many more instances than is being allowed for by our current leaders.

    diversity does not mean inviting to the table viewpoints which are simply manufactured and wished into existence. it means that anyone with a serious contribution is welcome. and really, that is what 'bipartisanship' is supposed to mean as well. but the way the term is thrown around these days, and the way in which it's used in this diary, may as well just call it 'natural flavors'. it means nothing. this crisis is quite literally several orders of magnitude beyond 9/11. we need to stop the bleeding, and give leeway on the very "far political left" for the president to try out moderate measures, watch them fail, and then try better ones. that is our only hope in this. compromising with the likes of Evan Bayh is not going to help anyone. trying to equate what these crazed governors are doing turning down direct federal aid to people (and this is now the dominant Republican tactic so 'bipartisanship' means meeting this mentality half-way) with reasonable behavior, makes any call for bipartisanship null and void from the get-go.

    the math is interesting to me, I love weird math problems. the ensuing political analysis, I didn't care for.

    Behold, the undead owl.

    by itsbenj on Fri Mar 13, 2009 at 05:23:02 PM PDT

    •  One thing that I wonder about is... (0+ / 0-)

      How much diversity, not just in ideology, but in socioeconomic status (not just current, but throughout their lives), personal experience, past careers, and everything else, Republicans have.  I mean, Republican politicians are disproportionately white, male, well-to-do, etc.  We don't see a lot of Jon Testers or Hilda Solises, John Lewises, or Barack Obamas in the Republican party.  Party ideologies aside (and really, Page hasn't looked into that, while he has looked into non-ideological aspects of diversity), the Democratic Party may be as strong as it is because it takes in people from every walk of life to know what's best for the American people.

      I gotta run, but I'll be back to take this up again soon.

      And check out Page's work.  I'm probably not doing him anything close to justice.

      •  Back, and available for further discussion (0+ / 0-)

        I'm still figuring all this out myself. Let me know if there's anything you're specifically interested/disagreeing with/otherwise affected by in some way and I shall do my best with it.

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site