Skip to main content

Good morning and Hugggggs to all who want them.

plf515 here, your associate professor of mathematical geekology. Thanks to NCrissieB for another opportunity to address the student body of Blogistan Polytechnic University

In a previous diary, I talked about the Kaplans and Math Circle; if you are interested in being trained as a math circle teacher, they have some spots available in their summer institute, July 5-11; more info here.

Today, I discuss the infamous Monty Hall problem - and review a book.

This is, by FAR, the most contentious math problem ever.  It depends on precise wording, but even then, is counter-intuitive.  

Join me below the fold, as we attempt to unravel it.

Here is how I will proceed:

  1.  Statement of the problem and brief notes on history
  1.  The answer
  1.  Intuitive approaches
  1.  Monte Carlo/computer programming approaches
  1.  A formal proof
  1.  A book review (beyond what's in the diary already)

and

  1.  A special bonus question.

I'll mention right up front that the book was sent to me by Oxford University Press, it's called the Monty Hall Problem, the author is Jason Rosenhouse, it's due out in early July, and I liked it a lot.  If you like this diary, I think you'll like the book

STATEMENT OF THE PROBLEM
You are on a game show.  You are presented with three doors.  Behind one of them is a car, behind the other two are goats.  You get to choose a door.  But before you open it, the host (Monty Hall), who knows where the car is, opens one of the other doors.  He always opens a door with a goat.  If both of the unchosen doors have goats, he picks one at random.  The car was placed randomly, with 1/3 chance for each door.

You are then offered a chance to change doors, or stick with your choice.

What should you do?

BRIEF HISTORY
This problem has been around a while, but it got famous when Marilyn vos Savant (self billed as the person with the world's highest IQ) wrote about it in Parade magazine.  She got it right, although her explanation wasn't 100% on the money.  She then got tons of mail.  Angry letters saying how stupid she was, bemoaning the state of education, and so on.  Some came from mathematicians, and some of the most vituperative ones were from mathematicians.  They were all wrong.

THE SOLUTION
You should switch.  If you switch, you have 2/3 chance of winning the car.  If you stick, you have 1/3 chance.

This is counter-intuitive to nearly everyone.  Even one of the greatest mathematicians (Paul Erdos) got it wrong.  But, I guarantee you, the solution is correct.

INTUITIVE APPROACHES
First approach - a video

elropsych was nice enough to forward me two videos that he uses with his AP Psychology students.
as he puts it

This one is animated, kind of childlike, but does a good job of presenting a graphic organizer that breaks down the probabilities, and their reversal, inherent in the problem.
http://www.youtube.com/...

This is a montage of 2 popular treatments of the problem, one from the tv show Numb3rs, the other from the movie 21. Neither clip explains the problem as well as the cartoon above, but I use them as entry points to begin the discussion as many of my students have either seen the show or movie, or at least heard of them.
http://www.youtube.com/...

Then, I get them up and we act it out, using students in the room to fulfill all the moving parts of the problem. At this point, most seem to "get" the reversal of probabilities in switching after the first card is revealed.

Second approach - the 100 doors
This one apparently convinces a lot of people.  It didn't help me, but maybe it will help you.  

Suppose there are 100 doors, and Monty reveals 98 of them.  Would you  switch?  

Third approach - added information - where does the information go?
It's clear that when Monty opens a door, he gives you added information.  But what information?  In this, the classic version, he cannot tell you  anything about YOUR door, only the other two.  Your door stays at 1/3, but all of the other 2/3 is now in one door.  OTOH, in an alternative version, where Monty does NOT know where the car is, he MIGHT give you information about your door.  If he shows a car, it tells you your door has a goat.

MONTE CARLO / COMPUTER methods

Statisticians use the term "Monte Carlo" methods for simulations that use random numbers to generate answers.  Often, this is done on computer.  Many people have programmed the problem, and they all get the answer that swapping is good, and all are at close to 2/3 vs. 1/3 ... as close as can be expected.  People have also done this using playing cards and simulating by hand.  They take 2 red aces (goats) 1 black ace (car) and then have pairs of people be Monty and the contestant.  This also gives the answer that swapping is right.

FORMAL PROOF

Here I rely heavily on the book (I rely on the book everywhere, but more so here).

First, we need to define a sample space.  Whenever you do an "experiment" broadly defined, the sample space is everything that could happen.

if you flip one coin, the sample space is {H, T}
if you flip two coins the sample space is {HH, HT, TH, TT}
if you  flip one coin and roll one die, the sample space is
  {H1, H2, H3, H4, H5, H6
   T1, T2, T3, T4, T5, T6}

the various outcomes do not have to be equally weighted.  If you ask two kossacks who they voted for, and use Mc, Ob and Ot for McCain, Obama, Other, then the sample space is {McMc, McOb, McOt,
                                ObMc, ObOb, ObOt,
                                OtMc, OtOb, OtOt}

but I know which one I'd bet on happening! :-)

in the Monty Hall problem, three things happen: You choose a door, Monty opens a door, and the car is behind a door.  We can represent each of these with A, B, and C for the doors.  So, if you choose door A, Monty opens door B and the car is behind door C, we would write {A, B, C}

The sample space in the classic Monty Hall problem is
{ABC, ACB, ABA, ACA,
BAC, BCA, BAB, BCB,
CAB, CBA, CAC, CBC}

We can make things a little simpler by assuming you choose door A to start with.  Now the sample space is
{ABC, ACB, ABA, ACA}

note that some triples are impossible.  e.g AAB is impossible, because Monty never opens your door, and ABB is impossible because Monty never opens the door with the car.

Remember that the four outcomes do NOT need to be equally likely; in fact, here, they are not.

We are told that the car is equally likely for the car to be behind any door. The location of the car is the third item in the triple, so this means
P(ABC) = P(ACB) = P(ABA) + P(ACA)

each being 1/3.  Note that there are TWO ways for the car to be behind door A.  But the total probability for door A is 1/3.

We are also told that, when Monty can open either door, he chooses at random so:
P(ABA) = P(ACA)
and, since the total is 1/3, each of these is 1/6.

OK, so we have
P(ABC) = 1/3
P(ACB) = 1/3
P(ABA) = 1/6
P(ACA) = 1/6

when do you win by switching?  In the first two cases, total probability 2/3.  When do you win by sticking?  In the last two cases, total probability = 1/3.

Alternatively, we could look at the sample space after Monty opens a door.  Say he opens door B. Since he does this half the time, we halve the sample space, but we have to double the probabilities associated with the outcomes.  The sample space is now

{ABC, ABA}

P(ABC) = 1/3*2 = 2/3
P(ABA) = 1/6*2 = 1/3

you win more often by switching.

BOOK REVIEW

This is one diary and the book is 200 pages.  So, what else is in the book?  There's considerable detail about the origins of the problem and the huge outcry when vos Savant published the right answer.  There's extensive coverage of a lot of variations of the Monty Hall game (e.g. different probabilities, more doors etc).  Much more important, though, this is a (mostly successful) attempt to teach a course in probability theory through the use of the MH problem.  

Who should read the book?

I think it has a couple audiences.  First, if you are taking a formal probability course at university, this could be a good backup to your text.  OTOH, if you  are teaching such  a course, you could use this as a main text (I've never seen a probability text that is this much fun to read).  A course based on this book would cover a lot of the ground of a one-semester intro to probability course.

Probably few at daily Kos are in either of those groups.  Among the general population, I think this book could be read in two ways: First, you could read chapters 1, 2, 6, 7, and 8, and either skip 3, 4, and 5 or skim them.  (Chapter 4, in particular, will be heavy going).  Second, if you want to learn probability theory, you could read the whole book.  In this case, you'll want to read it more like a text book.

Speaking of chapters, here's the table of contents:

  1. Ancestral Monty
  1. Classical Monty
  1. Bayesian Monty
  1. Progressive Monty
  1. Miscellaneous Monty
  1. Cognitive Monty
  1. Philosophical Monty
  1. Final Monty

BONUS QUESTION
Suppose 500 mathematicians wrote to Marilyn vos Savant, and 450 of them said she was wrong.  What proportion of mathematicians in the US are mixed up about this?  (take the poll)

OK, I had bonus question for you, so I will try to give bonus answers, by throwing the floor open for statistics questions.  If I can answer briefly, I'll do so in comments; if it requires a long answer, I'll write a diary.  If I can't answer, I'll try to say why, and provide a reference.

UPDATE
BONUS from the comments, where Merrily1000 tells more about how vos Savant argued

I even wrote to her. She got the answer correct, but really botched the explanation. She didn't explain that the probabilities changed because opening the door was not a random event, so people were confused. That's why a lot of people thought the answer should be 50 percent probability on each unopened door after Monty opened one. It really would be 50 percent if the choice of what door to open was random, and therefore if the opened door sometimes disclosed the car. Monty knew where the car was, and he never opened that door, and that's the point.

If you have the car behind your door, which is one third of the time, it doesn't matter which door Monty opens, you still have it.

But two-thirds of the time you don't have the car, and Monty simply opens whichever of the other two doors doesn't have the car, so two-thirds of the time the unopened door holds the car.

After her column, when a lot of people wrote her, Marilyn wrote a second column supporting her first one, and then asked people to run the experiment, which she presented as what would be the true proof.

Originally posted to plf515 on Tue May 05, 2009 at 03:52 AM PDT.

Poll

see question in diary

11%23 votes
8%17 votes
3%8 votes
8%17 votes
68%143 votes

| 208 votes | Vote | Results

EMAIL TO A FRIEND X
Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags

?

More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

  •  No "Full Monty"? (19+ / 0-)

    "The dirty little secret,,,is that every republican politician wants Obama to fail,," rush limbaugh

    by irate on Tue May 05, 2009 at 04:02:44 AM PDT

  •  love the goat question (12+ / 0-)

    it's always interesting to see how people answer it, and even more so their explanation as to why.

    Sic gorgiamus allos subjectatos nunc -7.25 -8.15

    by mydailydrunk on Tue May 05, 2009 at 04:09:21 AM PDT

    •  I remember a great variation (4+ / 0-)

      which was published, I believe, in Scientific American or the Smithsonian mag back in the 90's. It examined the fundamental difference between the following similar questions:

      1. A woman has two children. One of them is a boy. What are the chances that the other one is also a boy?

      1 in 3

      and,

      1. A woman has two children. The eldest is a boy. What are the chances the other one is also a boy?

      1 in 2.

      two different answers.

      That was hard enough to grasp, but the article went on to describe how, if one is shown a photo with this mother and her two children (where one child is obscured by a giant sheepdog in the foreground) and asked again what the odds are of the other (hidden) child being a boy... well, the answer is the same as question #2 simply because the photograph permits identifying the children in space by having a leftmost and rightmost child, which is as valid a distinguishing characteristic as eldest and youngest.

      This is the same core issue as that described in the diary, right?

      •  Now with a little coffee in me... (3+ / 0-)
        Recommended by:
        TrueBlueMajority, plf515, NCrissieB

        I'm thinking my comment has nothing to do with the Monty Hall problem at all.

        But, damn, it was a great article.

        Great diary too, BTW.

      •  I'd like a link to the explanation (1+ / 0-)
        Recommended by:
        plf515

        I'm at work and really don't have time to puzzle that one out.

        Hige sceal þe heardra, heorte þe cenre, mod sceal þe mare, þe ure mægen lytlað

        by milkbone on Tue May 05, 2009 at 07:55:59 AM PDT

        [ Parent ]

        •  OK (2+ / 0-)
          Recommended by:
          plf515, milkbone

          The possible genders of her children in 1 are FF, FM, MF, MM.  The first one isn't possible, so the last three cases are left (and all equally likely).  In only one of those three cases are both children boys, so the odds are 1/3.

          In 2 the possible choices are only MM and MF since you know the first one is M, so it's 1/2.

          Yes, there are progressives in the rural South. 50 States.

          by Racht on Tue May 05, 2009 at 08:14:06 AM PDT

          [ Parent ]

        •  If she has two children... (2+ / 0-)
          Recommended by:
          plf515, milkbone

          ...there are four possible arrangements of the genders: GG, GB, BG, BB (G=girl, B=boy).  If one of them is a boy, GG is excluded as a possibility, leaving 3.  Of those, only one is two boys.

          The second part (eldest is boy, what's second), is equivalent to a stand alone consideration of the second child, where there is a 50% chance of each.

          •  Interesting (2+ / 0-)
            Recommended by:
            plf515, NCrissieB

            Obviously this is correct; the logic works. I'm still trying to puzzle out exactly how the substitution of eldest for one makes the logical difference, though. It just doesn't seem to be enough information to make that difference.

            Hige sceal þe heardra, heorte þe cenre, mod sceal þe mare, þe ure mægen lytlað

            by milkbone on Tue May 05, 2009 at 09:21:29 AM PDT

            [ Parent ]

            •  The sample spaces are different (1+ / 0-)
              Recommended by:
              NCrissieB

              There are 4 combinations, listing elder and younger

              BB
              BG
              GB
              GG

              now, if we are told "the eldest is a boy" that leaves
              two combinations

              BB and BG

              if we are told "one of them is a boy" then there are
              three combinations:
              BB
              BG
              GB

              •  Depends on how you read it. (2+ / 0-)
                Recommended by:
                plf515, milkbone

                There are eight possible combinations for the three binary variables: "two children," "each is a boy or a girl," and "sex of child is known or unknown":

                {Bb, bB, Bg, bG, Gb, gB, Gg, gG}

                I used a simpler notation than in my other comment below: the children are listed in birth order, and the Capital means the sex is known.  Of these we can eliminate those with a capital-G, because there the known child is a girl, and the problem states "one of them is a boy."  So the known child is a boy and we're down four combinations:

                {Bb, bB, Bg, gB}

                In plain language this translates to:

                1. The older child (whose sex is known) is a boy, and the younger child (whose sex is unknown) is also a boy.
                1. The younger child (whose sex is known) is a boy, and the older child (whose sex is unknown) is also a boy.
                1. The older child (whose sex is known) is a boy, and the younger child (whose sex is unknown) is a girl.
                1. The younger child (whose sex is known) is a boy, and the older child (whose sex is unknown) is a girl.

                Of those four, there are two where the other child is a boy (older and younger), and two where the other child is a girl (older and younger).  So it's a 50/50 guess on the other child.

                The 'gotcha' answer equivocates on birth order.  It ignores birth order where the sibling is a brother (boy-boy), but distinguishes birth order where the sibling is a sister (boy-girl, girl-boy).

                If you know "the older child is a boy," we can eliminate six of the eight original combinations, leaving only: {Bb, Bg}.  Again, it's a 50/50 guess on the other child.

  •  Good Morning plf515 and Krew! (11+ / 0-)

    It's a little too early for me to wrap my mind around this but interesting diary. I was already thinking of trying the out problem with the 3 cards. I will have to ponder this a bit more when I'm fully awake!
    Huggggggggggggs!

  •  Rosenhouse's book sounds fascinating. Thanks for (9+ / 0-)

    tipping it to us! I've been looking for a Father's Day present for the hubby, and this is it. I suppose it would be wrong to hand it over with an already-creased spine?

    Morning hugs, Peter and all. I really enjoy your math diaries. (and your book diaries, and...)

  •  So does the answer change because I (11+ / 0-)

    Know the answer?  If I know that switching will improve my odds, then why not choose the other door in the first place?

    Math -- my worst subject.  And I'm a scientist.  Go figure!

  •  Interesting diary, Peter :) (11+ / 0-)

    All the more interesting because a friend explained the Monty Hall problem to me last week (agreeing, btw, with your explanation).  I was beating my head on the wall, groaning, arguing flailing like mad against what I thought I remembered from studying probabilities....

    In all, it is a quite painful problem to wrap the mind around. I can understand why even mathematicians don't get it, because it's hard, hard, hard.  And it's so counter-intuitive that even now that I get it I don't like it.  And don't want to believe it.

    How's that?

    Good morning and hugs to all the Krew!

    The austerity you see around you covers the richness of life like a veil -- Anonymous

    by winterbanyan on Tue May 05, 2009 at 04:22:35 AM PDT

    •  Indeed, that is the common reaction (10+ / 0-)

      Probability is full of things that are very counter-intuitive, but the Monty Hall problem (and variations) probably hit at this harder than almost anything else.

      Our intuitions are just very bad at this, and it doesn't seem to matter how much training one has had, nor how mathematical one is.  

      •  It's like (2+ / 0-)
        Recommended by:
        plf515, NCrissieB

        somebody who knows how to count cards in blackjack. Their bets seem WAY out of line with the cards on the table, and sometimes they take hits when most people wouldn't, but they come out ahead.

        Actually, if you pay attention to the odds and the rules in a casino, you can play for a long time without losing much money.

        You just have to play the right games and make the right bets - and not expect to win a whole lot.

        If you go in with the object of breaking even, you play a lot differently than if you're trying to win.

        And if you want to play for a long time without losing much, you definitely don't play the slots.

  •  More about vos Savant, please (9+ / 0-)

    What did she get wrong in her answer, and what were some of the arguments used against her reasoning?  I rarely quite understood what she was talking about but was dazzled by her none the less.  

    I wonder if this would work on The Dating Game? Two losers, one winner. Of course, Monty wouldn't be there to reveal date/man #1 as one loser...

    •  vos Savant got the answer right (11+ / 0-)

      but she used intuitive arguments that don't really work all the time (although they work in this particular case).  

      The problem with intuitive arguments in probability is that it is so easy to go wrong.

    •  You want more about vos Savant? (9+ / 0-)

      She's not as smart as she thinks she is. LOL  I stopped reading her the day I saw the logical flaw in one of her arguments, and I ain't no Einstein.  I was less bothered by the fact that she was wrong, than that she told a reader he was wrong...when he was right.  (And no, I didn't take only my own opinion on this, because I don't have the world's hightest IQ!)

      I'm not surprised she didn't get this one quite right.  But then, I'm surprised anyone gets this one right....

      Hugggs, JF :)

      The austerity you see around you covers the richness of life like a veil -- Anonymous

      by winterbanyan on Tue May 05, 2009 at 04:34:55 AM PDT

      [ Parent ]

      •  Like you, I too would stubbornly, intuitively (9+ / 0-)

        probably stand pat with my first choice, but the animated video makes a lot of sense for choosing the other door.  I wonder how many Monty's, by any other name, have set me up to inevitably, intuitively make the wrong choices?

      •  Vos Savant has been spectacularly wrong (8+ / 0-)

        She wrote a book (I refuse to link to it) explaining that Andrew Wiles' proof of Fermat's Last Theorem is incorrect. In this book she mostly proved how little she understands about real mathematics.

        For example: She doesn't understand the distinction between "mathematical induction", a rigorous proof technique, and "inductive reasoning". In particular, she explains how inductive reasoning does not constitute formal proof, and concludes that Wiles' proof is inadmissible because it uses mathematical induction!

        How could someone so presumably intelligent write such an embarrassing book? My own theory is that she did it solely for the money. I just can't believe that she doesn't realize how little she understands of the math behind the proof.

        •  If she was spectacularly wrong then almost (2+ / 0-)
          Recommended by:
          plf515, Jimdotz

          every single mathematician has been spectacularly inept, because Wile is the only one to prove Fermat's theorem right?  I think there is some jealously here.  You can see the same raw emotion in the angry letters that poured into her office about the Monty Hall problem.

          Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

          by hestal on Tue May 05, 2009 at 05:17:01 AM PDT

          [ Parent ]

          •  Your first sentence does not follow at all (7+ / 0-)

            vos Savant claimed Wiles proof was wrong, and used a ridiculous argument to try to make that claim.  No mathematician did that.  

          •  That's silly (3+ / 0-)
            Recommended by:
            plf515, elropsych, NCrissieB

            If she was spectacularly wrong then almost every single mathematician has been spectacularly inept, because Wile is the only one to prove Fermat's theorem right?

            And Darwin was inept because he didn't know the mechanism of evolution?

            Mathematical input grows over time. Even though Fermat's Theorem is a simply stated number theory conjecture, Wile's proof involved hyperbolic geometry, a theory of knowledge not available to Fermat or many generations of mathematicians who followed him. Moreover, Wile used a great deal of previous work by others to build the final chain. He certainly did not go directly from Fermat's statement to final conclusion.

            Indeed, there is so little number theory in Wile's proof (which had mistakes initially, but which has been patched) that Vos Savant mistrusted it. Some mathematicians still have foundational problems with a proof that goes far afield of the original domain. (This is not to excuse Vos Savant.)

            If you don't know where you're going, any road will do.

            by exregis on Tue May 05, 2009 at 05:53:29 AM PDT

            [ Parent ]

          •  Two scenarios (6+ / 0-)

            I'll let you be the judge:

            Door #1: Marilyn Vos Savant, despite her lack of advanced degrees in Mathematics, hampered by commitments to her weekly column and CFO duties at Jarvik Heart, Inc., finds a fatal flaw in Wiles' 100-page proof.  Armed only with her stratospheric IQ and an intuitive grasp of elliptic curves, Galois groups, and Hecke operators, she devastatingly debunks the proof that consumed Wiles for years.

            Door #2: In a shameless, yet inept bit of self-promotion, Vos Savant takes aim at one of the more publicly visible "smart people", Andrew Wiles. (Remember that Vos Savant's debunking book came out only a few months after Wiles' proof.)  The operative theory: if you knock the smartest person from his or her pedestal, then you must be The Smartest Person in the World.

            In the spirit of this diary, I'll reveal two oblique clues to help you choose...

            Clue #1: In her book debunking Wiles, Vos Savant

            "argued that because "the chain of proof is based in hyperbolic (Lobachevskian) geometry," and because squaring the circle is considered a "famous impossibility" despite being possible in hyperbolic geometry, then "if we reject a hyperbolic method of squaring the circle, we should also reject a hyperbolic proof of Fermat's last theorem."

            Clue #2: Years ago, I worked for a physics professor who as a junior faculty member was assigned to reply to the steady trickle of correspondence from Einstein debunkers. It happens that most major physics departments receive thick manila envelopes from Very Smart People, who have proved the impossibility of Einstein's Special Theory of Relativity. (No, they never attack his work on the photoelectric effect or Brownian motion, and rarely get near General Relativity.) My advisor was responsible for politely digesting the single-spaced "proofs" (when possible), and replying gently.  Sadly, this process often led to further overstuffed manila envelopes arriving.

            So, Door #1 or Door #2?

            If you don't know where you are going, you will wind up somewhere else. Yogi Berra

            by Twin Planets on Tue May 05, 2009 at 06:18:38 AM PDT

            [ Parent ]

      •  not everyone who is smart can teach (1+ / 0-)
        Recommended by:
        plf515

        my problem with vos Savant (come on, is that her real name???) is that she is not good at explaining things to people who are not on her level.

        many smart people cannot teach.

        Politics is like driving. To go backward, put it in R. To go forward, put it in D.
        President Obama. Still a thrill to see that in print.

        by TrueBlueMajority on Tue May 05, 2009 at 07:39:32 AM PDT

        [ Parent ]

  •  Excellent diary ... and easiest explanation is: (20+ / 0-)

    The easiest explanation:

    You had only a 1/3 chance of guessing the right door correctly to start.  That means there's a 2/3 chance you guessed the wrong door.  Monty knows where the car is and opens a door to reveal a goat.

    Now, 2/3 of the time your initial guess was wrong and the other unopened door must be right.  Conversely, 1/3 of the time your initial guess was right and the other unopened door must be wrong.

    Thus it's twice as likely you'll win by switching.

    The critical piece of information is italicized above.  If Monty doesn't know where the car is and picks one of the other two doors at random, then 1/3 of the time he'll open that door to show you the car you've already lost.

    This raises the interesting question: what if you don't know whether Monty knows where the car is?  What if Monty opens one of the other two doors and reveals a goat, but you don't know whether he did that on purpose?

    Good morning! ::hugggggggggggs::

  •  My own generalization of the problem . . . (6+ / 0-)

    Original with me, though no doubt others have come up with it independently.

    In the original problem, switching doors gives you a 2/3 chance of winning. If, on the other hand, Monty opens a door at random, and the door he opens happens not to conceal the prize, then you have a 50-50 chance of winning whether you switch or not. (The diarist alludes to this variant but doesn't explicitly state the resulting probability.)

    Now consider the following two further variants. In each, we start with four doors and you pick one as your very own.

    Variant A:  After you pick your door, Monty (knowing where the prize is) opens a door that he knows does not contain the prize.  Then he opens one of the remaining two doors at random and the prize does not appear.

    Variant B:  After you pick your door, Monty opens one of the other three doors at random and the prize does not appear. Then he opens one of the remaining two doors, one that he knows does not contain the prize.

    In each case there are now two doors left. Question: Are the two situations identical? If so, should you switch doors, and what is your chance of winning?  If not, in which variant (if either) should you switch, and what is your chance of winning in each variant?

    Non-math nerds stop reading here.

    In general, start with N doors (behind one of which there is a prize) and a bit vector V of length N-2.  You pick a door.  Monty now processes V left to right: Each time he encounters a 0 he opens a door other than yours at random. Each time he encounters a 1 he opens a door other than yours that he knows doesn't conceal the prize. At the end there are two doors unopened. Given that (by chance) the prize has not been revealed, what is the probability that you will win if you switch, as a function of V?

    (The original Monty Hall problem is V="1", variant A is V="10", and variant B is V="01".)

    For even more generality we can start with N doors concealing k < N prizes and a bit vector of some length less than N-k. I don't think this makes things more interesting.

  •  what are the odds that two kids who have (19+ / 0-)

    three tasks getting ready for school (eat breakfast, brush teeth, get dressed) will find 647 different ways to be late for the bus? In my house, the odds come in at 100%

    Morning all! Hugggggs!
    Fascinating diary, Peter, but I can tell more than ever that I just don't have a math mind...

    "We have always known that heedless self-interest was bad morals; we know now that it is bad economics" FDR

    by theKgirls on Tue May 05, 2009 at 04:36:18 AM PDT

  •  I remember Marilyn's column (16+ / 0-)

    I even wrote to her. She got the answer correct, but really botched the explanation. She didn't explain that the probabilities changed because opening the door was not a random event, so people were confused. That's why a lot of people thought the answer should be 50 percent probability on each unopened door after Monty opened one. It really would be 50 percent if the choice of what door to open was random, and therefore if the opened door sometimes disclosed the car. Monty knew where the car was, and he never opened that door, and that's the point.

    If you have the car behind your door, which is one third of the time, it doesn't matter which door Monty opens, you still have it.

    But two-thirds of the time you don't have the car, and Monty simply opens whichever of the other two doors doesn't have the car, so two-thirds of the time the unopened door holds the car.

    After her column, when a lot of people wrote her, Marilyn wrote a second column supporting her first one, and then asked people to run the experiment, which she presented as what would be the true proof.

    Sheesh.

    The soup got more affectionate.

    by merrily1000 on Tue May 05, 2009 at 04:38:02 AM PDT

  •  first clip (9+ / 0-)

    Your first youtube clip is the first explanation that has made sense to me, although I feel I'm now trying to fool myself by thinking I've fully grasped the problem.
    I'm converted, but a bit like a religious convert who says, "It doesn't make sense, it's absurd, therefore I believe."

  •  Good morning plf and Krew! (7+ / 0-)

    I love your math diaries; I can really understand, which is unusual for me and math!

    Don't believe everything you think.

    by EJP in Maine on Tue May 05, 2009 at 04:40:54 AM PDT

  •  OT, but ... "Feliz Cinco de Mayo!" (11+ / 0-)

    May I be one of the first to wish everyone a "Happy Cinco de Mayo"

    CincoDeMayo

    Our nation's incomparable FLOTUS decided it was such a grand holiday, she began celebrating it a day early at the Latin-American Montessori Bilingual (LAMB) Charter school in Washington.

    Have you noticed that she just never runs out of hugs?

  •  Cards and Probability (10+ / 0-)

    There is an application of this argument to Contract Bridge.  Restricted Choice Theory observes that when trying to guess the likelihood of an opponent's holding, one must consider holdings where an opponent is free to choose her play and those in which an opponent is compelled to make a certain play.  Analagously here, if you have chosen the right door, Monty is unrestricted in which door he reveals as the zonk.  When you have chosen the wrong door, Monty is restricted in which door he can show you.

  •  A comment on the 100 doors method (5+ / 0-)
    Recommended by:
    sheba, npb7768, plf515, kurt, Jimdotz

    Here's a way of stating it that may be more persuasive.

    Suppose you play the lottery. Let's say it's a "choose 6 out of 50" lottery (no bonus ball). You buy a ticket, picking numbers 8-21-28-30-39-46.

    After the lottery, but before you see the winning numbers, I come to you and say "Hey, I looked up the winning numbers, and you know what?  Either you won, or the winning numbers were 15-19-31-37-41-42."

    So, do you think there's now a 50% chance that you won? Of course not. Your chances of winning are the same 1 in a zillion that they always were. Put another way, if you could now retroactively go back and play the numbers I mentioned, of course you'd do it!

    And yet I can always make a statement like this---I know the winning numbers, and if you did win I can name any other numbers I like.

    Now notice that this situation is exactly the Monty Hall problem with lots of doors. My statement is the equivalent of opening all the other doors except one, showing that they didn't win. I leave one door unopened, plus your original choice.

  •  I've NEVER understood that problem! (12+ / 0-)

    And I have a degree in math. Such is life when one is damned good at statistics and stinky at probability. (It's the difference between inductive and deductive reasoning, I think. Like buying a book after you've read a few pages versus having the book and predicting what a page will say.)

    And sorry, plf - I lost the explanation, too - somewhere along the line I stopped understanding the sample space (ABA? How'd they get into this - is vos Savant being sued? And is the CBC covering this?).

    ...sigh...

    I do believe the conclusion - that it's better to switch - because experimental data always show that result. But I guess that unless someone can come up with an explanation in words of one syllable or less, for the statistically secure but probabilistically inept, it's one of those things I'll always just have to accept. Grumble, grumble.

    Damned fine diary - at least I followed most of it!  :-)

    "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." --- Ed Howdershelt (Author)

    by SciMathGuy on Tue May 05, 2009 at 04:57:18 AM PDT

    •  I offered an easy explanation above. (10+ / 0-)

      There's a 1/3 chance you guessed correctly first, and a 2/3 chance you guessed incorrectly.

      Monty now knowingly and intentionally shows you an incorrect choice from among the two unchosen doors, by opening that door to show you a goat.

      There's still a 1/3 chance your original choice was correct - and Monty ruled out the opened door when he opened it - so there's a 2/3 chance the other unopened door is correct.

      Good morning! ::huggggggggggs::

      •  I think the biggest problem to understanding (3+ / 0-)
        Recommended by:
        plf515, SciMathGuy, NCrissieB

        the problem - certainly the biggest I had until about 15 minutes ago - is to thoroughly understanding the ground rules, in particular that Monty knows what's behind all three doors and is forced to choose a goat after you make your initial selection.

        Hige sceal þe heardra, heorte þe cenre, mod sceal þe mare, þe ure mægen lytlað

        by milkbone on Tue May 05, 2009 at 08:26:08 AM PDT

        [ Parent ]

      •  Ah, OK! Thanks, NCrissieB! (2+ / 0-)
        Recommended by:
        plf515, NCrissieB

        That does make sense! Now, if only I can remember it!

        "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." --- Ed Howdershelt (Author)

        by SciMathGuy on Tue May 05, 2009 at 02:39:15 PM PDT

        [ Parent ]

    •  Probability is good (7+ / 0-)

      if you're playing bridge, poker and other card games.  I used it a lot playing duplicate bridge.
      Watson's Play of the Hand by Louis H. Watson is the perfect primer for that.  OTOH I find using probability on primary decision making of important life matters (or even some that are not so important) fails me more often than not. BTW a good morning to all who are partaking in this Morning Feature; a special thanks to plf for filling in this morning.

      •  Thanks for the book lead ... (3+ / 0-)
        Recommended by:
        plf515, SciMathGuy, elropsych

        ... I've recently begun playing bridge fairly regularly (after having played very sporadically about 35 years ago and not at all since). I figure some basic probabilities would improve my chances greatly.

        Here's one fairly common situation for which I tried to work out the probabilities: You're declarer and hold the ace, king, and jack of trump. There are five trump out against you. Is it better to try the finesse or to play the ace, king hoping to drop the queen? (Opponents haven't bid so you have no information from them.) I think the finesse is the better play.

        Also, when you're not vulnerable and have nothing on, I think it's always better to play at game level rather than one below - even if you're short on points for game.

    •  Some Easy Ways to Understand the Monty (11+ / 0-)

      Here are some easy ways to "see" the solution:

      1. Pretend the game is that you are in the audience and making a bet on who is going to win, Monty or the contestant.  There are 3 doors, Monty lets the contestant have one.  Who are you going to bet on?  Monty, of course, he has 2 doors to one.  Then Monty turns around and says, "Hey let's switch, you can have 2 doors, I will take one."  Who are you going to bet on?  The guy who takes the switch.
      1. Instead of Monty saying, "I am OPENING door number 1" think of Monty saying, "I am GIVING you Door Number 1 if you also switch doors."  Now you have two doors and Monty has one.  I would take that offer.
      1. Take 3 playing cards and start playing the game by yourself.  You can see right away how it works.

      Here is some more interesting history.  Marilyn Savant was definitely not considered as "cool" or "hip" in those days, more of a huckster.  There was another guy named Cecil Adams who had a very cool and hip column called The Straight Dope, which was sort of a "rival" of Savant that appeared in alternative newspapers.  When she came out with the Monty column, he jumped all over her and his fans (including me) were very happy.  Then when it turned out Savant was correct, Adams wrote another column trying to make his amends and he put in another logic problem with an incorrect answer.  Adams never seemed the same after the Monty fiasco.  He seemed to become very bitter about everything.

  •  Mathphobe bids all a hail, farewell and smooooooo (5+ / 0-)
    Recommended by:
    Orinoco, plf515, Jimdotz, elropsych, NCrissieB

    ...........ches.
    also hugs.

  •  I confess, I'm one of those people (8+ / 0-)

    who was unable to grasp the concept until the person explaining to me used "the hundred doors" whereupon I grasped the idea immediately.
    I know it's been commented upon endlessly, but it's still marvelous to me how some approaches succeed with some and not others.

    Alito. Kennedy. Roberts. Scalia. Thomas.
    More important than ever: ERA NOW!

    by greeseyparrot on Tue May 05, 2009 at 05:07:47 AM PDT

  •  I think that the Monty Hall problem has (9+ / 0-)

    importance for another reason.  It shows that our intuitions can be wrong, but because they are our intuitions, our common sense, we "feel" that they are true and won't easily give them up.  "Common sense," is often wrong and that is one of the reasons that our educators originally decided to teach math to everyone in the public schools.  They were trying to teach them how to "think" not "feel" their way through life.

    In addition to having no evolved intuition for probabilities, we have another failing that really hurts, and that is the value of time.  I used to introduce the idea to my students in this way.  I gave them the formula d=rt, where d=distance, r=rate, and t=time.  We would do a few simple calculations and then I would give them this problem:

    I drive two miles to work every morning.  One morning I drive the first mile at an average speed of 30 miles per hour.  How fast will I have to drive the second mile in order to average 60 miles per hour for the entire trip?  I would give them a couple of minutes to think about the problem and when most had lifted their gaze I would give them some choices on the board.  Most of them would vote for 90 miles per hour as the correct answer.  Almost no one would protest that I had omitted the correct answer from my choices.

    So we have two blind spots in our intutitions and they do hurt us in our lives.  Probabilities are almost always "felt" to be in our favor and so is time -- especially when we are young.  

     

    Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

    by hestal on Tue May 05, 2009 at 05:12:22 AM PDT

    •  Cool (4+ / 0-)
      Recommended by:
      hestal, Orinoco, plf515, Jimdotz

      Nice example.

      One place the reliance on "feel" really hurts comes in investments, when ownership or a previous choice almost always makes one tend to give too much weight to "sticking".

    •  Umm ... poor definition of "intuition." (4+ / 0-)
      Recommended by:
      terrypinder, Orinoco, plf515, elropsych

      Intuition is "invisible reasoning."  The quality of our invisible reasoning is limited by the same factors that limit our visible reasoning: education and experience.  It's not that "intuition" is often wrong, but that our reasoning is often wrong if we're analyzing a situation for which we lack the education and experience to do it well.

      If someone says "It doesn't matter whether you switch, because now there are two doors and you're either right or you're wrong," that's not really "intuition" anymore.  They've made their reasoning visible ... and it's still wrong.  Specifically, they haven't weighed the information gained by knowing that Monty knows where the car is and will always show you a goat.

      It's like someone arguing that 1+1=1 because "I have one lump of mashed potatoes, and I drop another lump of mashed potatoes on it, but it's still one lump of mashed potatoes."  That's not intuition; it's visible reasoning.  But it's being done without some important information: the amount of mashed potatoes in the lump(s).

      Good morning! ::huggggggggggs::

      •  Ummm ... selective use of definition in (0+ / 0-)

        order to find a way to disagree with me.

        My dictionary, Webster's Third New International Dictionary, published by the Encyclopedia Britannica, gives these choices (vol. II, page 1187):

        1 a obs: the act of looking upon, regarding, examining, or inspecting b archaic: the act of contemplating or considering:CONTEMPLATION, CONSIDERATION c obs: a view, regard, or consideration of something as an ulterior goal or acquisition, 2 a: the act or process of coming to direct knowledge or certainty without reasoning or inferring: immediate cognizance or conviction without rational thought : revelatiion by insight or innate knowledge : immediate apprehension or cognition b: knowledge, perception or conviction gained by intuition (trusting ...  to what are called ~s rather than reasoned conclusions - A.C. Benson) c: the power or faculty of attaining to direct knowledge or cognition without rational thought and inference d: in Bergsonism a form of knowing that is akin to instinct or a divining empathy and that gives direct insight into reality as it is in itself and absolutely e: quick and ready insight (with one of her leaps of ~ she had entered into the author's soul - Edith Wharton)

        The definitions listed in 1 are either obsolete or archaic.  Your definition is invisible to the authors of this compehensive dictionary, and because you start with a faulty premise, I ignored the rest of your comment.

        Please keep your hugs to yourself.  You are being entirely too personal for my comfort.

        Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

        by hestal on Tue May 05, 2009 at 06:04:03 AM PDT

        [ Parent ]

        •  "please keep your hugs to yourself" (6+ / 0-)

          What an entirely disagreeable person you are.  You may well have been the smartest person in the room your entire life, but no one will ever notice while you're being rude.  I suggest reading Dale Carnegie's How to Win Friends and Influence People, it was written with you very much in mind.

          This sig line was taken by the Rapture.

          by Maimonides on Tue May 05, 2009 at 06:22:37 AM PDT

          [ Parent ]

          •  Please keep your anger to yourself. (0+ / 0-)

            Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

            by hestal on Tue May 05, 2009 at 06:24:48 AM PDT

            [ Parent ]

            •  Um . . . Pot, kettle calling, says your (1+ / 0-)
              Recommended by:
              plf515

              overreacting to everything.

              This sig line was taken by the Rapture.

              by Maimonides on Tue May 05, 2009 at 06:26:03 AM PDT

              [ Parent ]

              •  I am not angry. I am amused, but (0+ / 0-)

                also saddened at the irrationality of so many people when it comes to dealing with a smart woman.

                Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

                by hestal on Tue May 05, 2009 at 06:27:01 AM PDT

                [ Parent ]

                •  Faulty assumption: we care about her gender (1+ / 0-)
                  Recommended by:
                  plf515

                  results in faulty output.  

                  Again, have a nice day, and when you calm down I really hope you take the time to read Dale Carnegie.

                  This sig line was taken by the Rapture.

                  by Maimonides on Tue May 05, 2009 at 06:28:51 AM PDT

                  [ Parent ]

                •  BTW, women are responding to you. (1+ / 0-)
                  Recommended by:
                  plf515

                  But you're ignoring them . . . because they undermine your thesis?

                  This sig line was taken by the Rapture.

                  by Maimonides on Tue May 05, 2009 at 06:29:56 AM PDT

                  [ Parent ]

                  •  I don't know if they are women, but I am (0+ / 0-)

                    saddened that misogyny is not the only cause of irrational anger.

                    Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

                    by hestal on Tue May 05, 2009 at 06:33:56 AM PDT

                    [ Parent ]

                    •  So your thesis is Adamantium, untestable (2+ / 0-)
                      Recommended by:
                      plf515, winterbanyan

                      and impervious to reflection on your part.

                      That's the truly sad thing here.

                      This sig line was taken by the Rapture.

                      by Maimonides on Tue May 05, 2009 at 06:35:08 AM PDT

                      [ Parent ]

                      •  Nice try, but you have been reduced to (0+ / 0-)

                        name-calling and you are no longer of any interest to me.  

                        Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

                        by hestal on Tue May 05, 2009 at 06:37:01 AM PDT

                        [ Parent ]

                        •  Person accusing me of misogyny wants to quit now! (2+ / 0-)
                          Recommended by:
                          plf515, winterbanyan

                          I've called you rude, vindictive and accusatory, but the record (and other commenters responses) seem to verify that.

                          I'd hazard to guess that Marilyn would like nothing more than for people like you to avoid defending her.  Thus far you've yet to respond to a single point about the actual problem that I've made, preferring to keep to accusations and, well, rudeness.

                          I haven't ever called you a name.

                          This sig line was taken by the Rapture.

                          by Maimonides on Tue May 05, 2009 at 06:48:02 AM PDT

                          [ Parent ]

                      •  No, adults don't have to become angry (0+ / 0-)

                        over mathematical logic.  But immature hotheads do.

                        Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

                        by hestal on Tue May 05, 2009 at 06:41:38 AM PDT

                        [ Parent ]

                        •  Therefore hestal = immature hothead. nt (2+ / 0-)
                          Recommended by:
                          plf515, winterbanyan

                          This sig line was taken by the Rapture.

                          by Maimonides on Tue May 05, 2009 at 06:58:56 AM PDT

                          [ Parent ]

                          •  Still haven't regained your composure, huh? (0+ / 0-)

                            Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

                            by hestal on Tue May 05, 2009 at 07:02:54 AM PDT

                            [ Parent ]

                          •  I don't understand how this thread got started (4+ / 0-)

                            meaning, i'm not sure where you made the jump from a criticism of Vos Savant to misogyny. It really isn't there and is exceptionally illogical. Not to mention most of the people you are arguing with (for no real reason) are women themselves.

                            Perhaps everyone should just walk away and go do something else.

                            (0.12, -3.33) disagreement does not automatically render one a shill. duh.

                            by terrypinder on Tue May 05, 2009 at 07:08:13 AM PDT

                            [ Parent ]

                          •  There was a reason for my comments. (0+ / 0-)

                            If one reads the actual comments that Savant got, which she published on her website, then one can only conclude that misogyny was at work.

                            Most of those who were offended and who were downright rude to her were men.  Many were chairmen of university math departments.  Now either Savant deliberately selected only rude letters from males to publish, or most mathematicians are males, or only males got angry.  Furthermore in my world of mathematicians at the time I solicited reactions from colleagues and found that the males were overwhelmingly hostile to her.  One of them, for example, had advanced degrees in math and physics and was the head of reasearch for a very large electronics company and his immediate reaction, and angry reaction, was that she was dead wrong and that she had done much harm to mathematics teaching everywhere.  Three days later he called me to sheepishly admit that she was right and he was wrong.  

                            The argument still boils today.  You can see here that heat.  I have been an observer of this blog and some others for years now, almost from their inception, and they are fruitful areas to gather data about human behavior and about how anonymity permits unreasonable discourse to flourish, and about how anger is freely embraced.  

                            And these blogs are full of examples where people make fools of themselves in a special way.  This old adage is proven true time and again: "It is not what you don't know that makes you look silly, but it is what you know that just ain't so that does trick."

                            For example, misogyny, as it is used here is taken to mean a hatred of women by males, but that is not the definition of misogyny.  It simply means a hatred of women no matter the gender of the hater.  So both males and females can be misogynists.

                            I am making this study to see how the Internet can be employed to replace certain important national institutions.  One of these is political parties.  Where else can one find such examples of Interet political discourse, with an emphasis on political parties, than here and others of similar kind.

                            The evolution of these blogs has been fascinating and they are improving, but there are several important factors that are used to control debate, in a good way, that are absent from them.  As a consequence good ideas are lost in the heat of emotion.  I see it all the time.  There is more to this process that I am following but I think I should stop here.

                            Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

                            by hestal on Tue May 05, 2009 at 07:28:26 AM PDT

                            [ Parent ]

                          •  So this is all about you. Makes sense now. nt (0+ / 0-)

                            This sig line was taken by the Rapture.

                            by Maimonides on Tue May 05, 2009 at 07:31:50 AM PDT

                            [ Parent ]

                          •  You were the one who suggested (3+ / 0-)
                            Recommended by:
                            plf515, Maimonides, kktlaw

                            the limited definition, to wit:

                            Secondly you are so quick to forgive the men, yes men, who wrote in and castigated her in the most hateful terms, telling her that she was wrong, not unclear, but flat wrong.  If she had been a man the writers would have confirmed the correctness of her ultimate answer and they would have offered better explanations than his.  That is how gentlemen behave, but gentlemen, unfortunately do not behave like gentlemen when it comes to admitting that a woman, a WOMAN for god's sakes, is smarter than they.

                            And the final proof of my statements is that when such ungentlemanly persons have their rudeness pointed out to them they become uncontrollably angry and start SHOUTING.

                            I wasn't going to reply to you again, but I couldn't let this pass.  If your mode of study is to come on a blog, hijack the thread with baseless accusations, and then excuse yourself because we here are just bugs under your microscope, you may have to start eating doughnuts for breakfast.  You insulted my friends.

                            Basta!

                            The austerity you see around you covers the richness of life like a veil -- Anonymous

                            by winterbanyan on Tue May 05, 2009 at 07:39:23 AM PDT

                            [ Parent ]

                          •  You are incorrect. (0+ / 0-)

                            If you read the thread beginning with my first comment you will see that Mainmonides was the first to suggest that only men hate women.

                            I wrote: "I am not angry. I am amused, but also saddened at the irrationality of so many people when it comes to dealing with a smart woman."

                            Maimonides, in attack mode wrote: "BTW, women are responding to you. But you're ignoring them . . . because they undermine your thesis?"

                            Clearly Maimonides assumed that when I said "many people" I meant men.  He decided on his own, without any prompting from me, that my "thesis" was that only men have trouble dealing with a smart woman.  I did not say it, I did not imply it.  Maimonides made it up and you bought into it -- but he may be one of your friends so giving your support was natural.  I forgive you.

                            But I did pile on.  I wanted to see if anyone else would point out this leap to a conclusion.  But no one did.  The narrative was already established.  I was the black hat.  Apparently a lot of these people are "friends" in the Internet sense, and the gang effect came into play.  The whole process was routine.

                            So, winterbanyan, you, too have leaped to an incorrect conclusion.

                            I don't know what "Basta!" means, so I will just say Ciao.

                            Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

                            by hestal on Tue May 05, 2009 at 07:59:39 AM PDT

                            [ Parent ]

                          •  I was responding to your attack by noting (1+ / 0-)
                            Recommended by:
                            arrows theorem

                            that women were disagreeing with you about the misogyny.  My background in feminist anthropology leads me to believe that it is often best to let women weigh in on whether misogyny is present.

                            Which has nothing to do with my original comment that you've blithely ignored for ages now.

                            This sig line was taken by the Rapture.

                            by Maimonides on Tue May 05, 2009 at 08:28:31 AM PDT

                            [ Parent ]

                          •  You are the first to raise the issue of (0+ / 0-)

                            "misogyny."  I did not say it, you did.

                            Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

                            by hestal on Tue May 05, 2009 at 08:33:09 AM PDT

                            [ Parent ]

                          •  You strongly implied that Maimonides (2+ / 0-)
                            Recommended by:
                            plf515, Maimonides

                            was misogynistic here:

                            Your stubborness has been shared by many many other die hards who just can't accept being caught out by a woman.

                            If you can't accept the implications of your words, perhaps you should be more careful in choosing them. I have read this entire thread and have found your arguments to be illogical, emotion-based, and needlessly provocative.

                          •  Thank you! (2+ / 0-)
                            Recommended by:
                            plf515, arrows theorem

                            I've always wanted to be defended by a theorem.

                            This sig line was taken by the Rapture.

                            by Maimonides on Tue May 05, 2009 at 08:36:35 AM PDT

                            [ Parent ]

                          •  I feel sorry for you. (2+ / 0-)
                            Recommended by:
                            plf515, Maimonides

                            You are stuck in an argument with someone who only seems to care about the appearance of being logical and intelligent. And the "bugs under a microscope" comment is one of the oldest tricks in the try-to-prove-I'm-smarter-than-someone-on-the-internet book.

                          •  It's also an indicator for a broad spectrum (1+ / 0-)
                            Recommended by:
                            plf515

                            of social illnesses, not the least of which is sociopathy. Female sociopaths are rarely violent, but they love to play with other people's minds.

                            This sig line was taken by the Rapture.

                            by Maimonides on Tue May 05, 2009 at 08:51:08 AM PDT

                            [ Parent ]

                          •  I put a label to it, see below. (1+ / 0-)
                            Recommended by:
                            plf515

                            You leveled the accusation, I just gave it a term.

                            This sig line was taken by the Rapture.

                            by Maimonides on Tue May 05, 2009 at 08:36:03 AM PDT

                            [ Parent ]

                          •  How this actually came down (1+ / 0-)
                            Recommended by:
                            plf515

                            I said that she stated the problem wrong (fact) and is overrated (opinion).  You responded:

                            Does her IQ rate higher than yours?  Is this jealousy talking?  She has given hundreds of problems over the years, and this is the only mistake?  And she got the answer right, right?  You just didn't like her explanation?

                            I responded, reasonably, that GIVEN HER EXPLANATION, her answer was originally incorrect (fact), and that her tree falling in the woods explanation was a cop-out (opinion).

                            You responded:

                            Your stubborness has been shared by many (0+ / 0-)
                            many other die hards who just can't accept being caught out by a woman.  Go to Savant's site to see some of the letters she received.  You will see yourself there

                            And right there, without any NEED to know my gender, you ignored the two times I brought up her original mistake in favor of saying that my comments were because she's a woman.  Implicitly, because of misogyny.

                            Sorry, the record, YOUR record, stands.

                            This sig line was taken by the Rapture.

                            by Maimonides on Tue May 05, 2009 at 08:35:11 AM PDT

                            [ Parent ]

                          •  And then . . . (1+ / 0-)
                            Recommended by:
                            plf515

                            I asked you to apologize, which you didn't

                            and then you went on to bizarrely accuse me of

                            You are wrong on so many levels. (0+ / 0-)
                            Misogyny is commonplace in the world, and especially in math and science.  (I never said it wasn't)

                            Secondly (there was no firstly) you are so quick to forgive the men, yes men, who wrote in and castigated her in the most hateful terms, telling her that she was wrong, not unclear, but flat wrong. (I never brought them up at all, or their gender.  That's all you.) If she had been a man the writers would have confirmed the correctness of her ultimate answer and they would have offered better explanations than his.  That is how gentlemen behave, but gentlemen, unfortunately do not behave like gentlemen when it comes to admitting that a woman, a WOMAN for god's sakes, is smarter than they. (And for fun you finish off with a broad stereotype, bravo!)

                            And the rest is you going off without ever getting to the main point, the thing I find interesting: the difference between a statistical problem and a problem of information theory.

                            We could discuss sexism in the maths and sciences.  It could be a healthy debate.  My background in feminist anthropology might have informed you, and as an anthrolopologist I would have been interested in your observations as a woman in that world.  But alas, you chose an enemy, and made one.

                            This sig line was taken by the Rapture.

                            by Maimonides on Tue May 05, 2009 at 08:48:14 AM PDT

                            [ Parent ]

                          •  i'm sorry (4+ / 0-)

                            but the criticism she recieved on her site regarding her solution to this problem has nothing to do with the criticism that Maimondes and winterbayan, among others, offered here.

                            you've made the assumption that those criticizing her are a.)jealous or b.) sexist without any evidence other then what she received ages ago in response. You've also made the assumption that people are angry. I can assure you Maimondes was having fun as logical reasoning sans emotional input is one of his big things (or, at least from following him through the blog over the last months and years, that's what I get from him).

                            You need to show a lot more evidence that the people posting here in criticism of Vos Savant are jealous angry misogynists. That seems to be the crux of your claim, and there's absolutely no evidence of such.

                            (0.12, -3.33) disagreement does not automatically render one a shill. duh.

                            by terrypinder on Tue May 05, 2009 at 07:54:08 AM PDT

                            [ Parent ]

                          •  You will have to read the entire thread, (0+ / 0-)

                            and it is a long one, and you will have to have an open mind, but I plead not guilty of the charges you have made.

                            Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning.

                            by hestal on Tue May 05, 2009 at 08:01:04 AM PDT

                            [ Parent ]

                          •  I AM NOT A VULCAN . . . er, ok, I am. (1+ / 0-)
                            Recommended by:
                            terrypinder

                            Live long and prosper.  Dammit.

                            This sig line was taken by the Rapture.

                            by Maimonides on Tue May 05, 2009 at 08:26:00 AM PDT

                            [ Parent ]

                          •  Now I'm just needling you because it's easy. (1+ / 0-)
                            Recommended by:
                            plf515

                            I started by being reasonable, even gave you a second swing at my explanation.  Then I tried to convince you that you were overreacting.

                            Now I'm just poking with the poor frothing sod with the aggression disorder.

                            This sig line was taken by the Rapture.

                            by Maimonides on Tue May 05, 2009 at 07:11:11 AM PDT

                            [ Parent ]

            •  If you want people to not be angry with you (2+ / 0-)
              Recommended by:
              Maimonides, MaryinHammondsport

              stop being obnoxious.

    •  Cute problem (4+ / 0-)
      Recommended by:
      hestal, indybend, Orinoco, plf515

      I teach Physics and I have another cute one:

      I travel 100 miles at 50 mi/hr, on the return trip I travel at 100 mi/hr. What is my average speed for the entire trip?

      99+% of students initially say it is 75 mi/hr, and they are wrong!

      See for yourself why this is so:

      Average speed= total distance/total time

      "The authorities aren't interested in the truth, merely in authority."

      by Fastrudy on Tue May 05, 2009 at 05:45:22 AM PDT

      [ Parent ]

    •  If you drive the first mile at 30mph (4+ / 0-)
      Recommended by:
      hestal, Bob B, Orinoco, NCrissieB

      then you would have to drive at infinite speed to average 60 for the whole trip.

      30 mph for 1 mile takes 2 minutes.
      60 mph for 2 miles takes 2 minutes

      so you have no time left to drive the 2nd mile.

    •  Misdirection (5+ / 0-)

      The kids' attention is focused on the miles per hour number, 30, 60, 90, and it makes sense that if you want 60 to result from averaging 30 and something else, that something else should be 90.

      If you take a look at the time involved in a two mile trip, you want an average speed of 60 mph, means you want to make the trip in 2 minutes. Unfortunately, by poking along at 30 mph for the first mile, you've used up your two minutes. You're toast. There's no possible way to travel that second mile in zero time, which you'd have to do to arrive at work two minutes after starting out.

      Good morning! and ::hugggggggs:: to the Kula Krew, our intrepid associate professor of mathematical geekology and all. :)

      "You can't get something for nothing...It's time to stop being stupid." Bob Herbert

      by Orinoco on Tue May 05, 2009 at 05:58:50 AM PDT

      [ Parent ]

    •  that is a great math exercise! (1+ / 0-)
      Recommended by:
      plf515

      an excellent challenge to lazy thinking. the wrong assumption here is presuming that when calculating the mean, (a+b)/2, that a=30. if you do the arithmetic correctly, you see that a=1/30 and the proper equation is (1/2)[(1/30)+(1/x)]=(1/60).

      freedom isn't free, but it isn't dumb either.

      by astro on Tue May 05, 2009 at 06:38:41 AM PDT

      [ Parent ]

  •  The only chapter title missing is (6+ / 0-)

    Full Monty.

    Somebody had to say it!

  •  Thanks again. (5+ / 0-)

    I've been looking forward to this since you promised it.  My head was okay with it, but this helped counter the counter-intuitiveness.  I could actually explain it to others at this point.  Muchas gracias.

    They only call it class warfare when we fight back.

    by rb608 on Tue May 05, 2009 at 05:16:29 AM PDT

  •  In the New York Times article (6+ / 0-)

    here, where John Tierney interviews Monty Hall, Monty points out that he doesn't have to offer a deal at all. If your first pick has a goat, he may just give you the goat. If you pick the car, he may try to con you into switching by making you think the car is behind another door. He says, if the host offers you money to switch, take the money; it's the only sure thing.

    •  Well, that's a different problem altogether (2+ / 0-)
      Recommended by:
      plf515, NCrissieB

      The real problem here is for Monty: if he always gives a goat when the contestant picks a goat door, then the contestant should never switch if offered the chance, because the only time Monty offers a choice is when they've picked the car. If people know Monty's behavior is consistent, they will never switch and Monty will give away one car per three contestants, based on their original choice of one door of three.

      So Monty has to figure out how often to offer a choice when the contestant has picked a goat in order to lower the odds that he gives away a car.

      "You can't get something for nothing...It's time to stop being stupid." Bob Herbert

      by Orinoco on Tue May 05, 2009 at 06:13:45 AM PDT

      [ Parent ]

  •  Most difficult question: (5+ / 0-)

    Where do you get the time?

    (Thanks for this diary. Very interesting. But I'm gonna have to look at it some more because even though I think I get it, I'm still not sure I believe it. Fortunately, I'm retired so I have the time.)

  •  Saturday's Kentucky Derby (3+ / 0-)
    Recommended by:
    plf515, Jimdotz, NCrissieB

    skewed the probabilities don't you think?

    •  Nope (3+ / 0-)
      Recommended by:
      Orinoco, Jimdotz, NCrissieB

      Being 50-1 against doesn't mean it WON'T happen, just that it's unlikely.  But there have been a lot of horse races!

      To show that the odds are wrong you'd have to look at a large set of races, the odds, and the winner.  I think the odds are pretty close to right, but not exactly.

      The reason not exactly is that these odds aren't really probabilities at all, they are proportions of the betting pool that bet a certain way.  

      •  Common problem in historical analysis. (3+ / 0-)
        Recommended by:
        terrypinder, plf515, winterbanyan

        "Event E happened."

        So what is the probability that Event E would happen?  Umm ... well ... uhh ... not zero.  We know it's not zero, because it did happen.  As for what its actual probability was, in most cases we neither know nor can know.

        What was the probability of Mine That Bird winning the Kentucky Derby?  Very close to 1.0 - barring injury or a jockey mistake - over the last 1/8 of a mile.  How about on the back stretch?  How about right before the bell rings and the announcer says "Annnnnd they're off?"

        We know what the betting odds were, and those reflect the best guesstimates of people who supposedly know something about horses and horse racing.  But we also knew what the betting odds were on an economic meltdown - the futures markets last summer - and those reflected the best guesstimates of people who supposedly know something about the economy.

        Oops.

        And OOPS.

    •  Derren Brown came up with a system (1+ / 0-)
      Recommended by:
      plf515

      Watch it on YouTube (in 6 parts). He gives a woman 5 (6?) winning horse tips in a row. Yes, really!

      http://www.youtube.com/...

      Deranged neoconservative militarism isn't the solution to nuclear proliferation; it's a cause. -- Glenn Greenwald

      by factbased on Tue May 05, 2009 at 10:55:46 AM PDT

      [ Parent ]

  •  I'll simply use the Jimmy Buffett method: (3+ / 0-)
    Recommended by:
    plf515, Jimdotz, NCrissieB

    "My whole world lies waitin' behind
    Door Number Three"

    When in doubt, tweak the freeqs.

    by wozzle on Tue May 05, 2009 at 05:20:05 AM PDT

  •  Treasure Chest (7+ / 0-)

    I was actively showing a group of people this very problem about a month ago. They did not believe me, so we played a simulation with three cups, a quarter and two pennies. It only took ten trials to convince them.
    A similar problem appeared in the Dell Science Series book, 'Lady Luck', in the early 1960's. It was a three card game called "Treasure Chest" and was devastatingly counter-intuitive. Whenever I needed some cash in college, I would engage some local in a tavern to play this with me at $1.00 per game. By the time they figured out that they were not going to win, I had enough spending money!
    Keep teaching math here, because as soon as most States approved a Lottery, the State Education Dept. watered down (or eliminated) the probability section of High School Math instruction.

    "The authorities aren't interested in the truth, merely in authority."

    by Fastrudy on Tue May 05, 2009 at 05:20:09 AM PDT

  •  Other reasons it is counterintuitive (7+ / 0-)

    This problem is definitely not intuitive for most people, but part of the problem is that it is often not described correctly. I have seen this presented many times where it was not made clear that Monty is required to always open a door after you have made your choice. This needs to be stated very clearly because everyone who has seen the real show knows perfectly well that the real Monty does not behave this way; the real Monty may very well behave differently based on whether or not you have made the correct initial choice.

    •  The key to the problem is to understand that the (5+ / 0-)
      Recommended by:
      raboof, Dave in RI, Orinoco, plf515, NCrissieB

      host must, by the conditions of the problem, always open a goat door after the contestant has made his initial choice.  

      If one of the two unselected doors had accidentally swung open, revealing a goat, or if the host had flipped a coin and opened a door according to the coin toss, then switching wouldn't improve the contestant's chances.  

      It isn't so much that our "intuition" leads us astray as that many of us have not been trained, or trained ourselves, to think carefully and rigorously about probability problems.  Such thinking is a learned skill, and if you have acquired it, the Monty Hall problem is elementary.

      •  Only partly right (3+ / 0-)
        Recommended by:
        Orinoco, NCrissieB, Toon

        it is certainly  true that you have to state the problem clearly.

        But, even with that, almost no one thinks that switching is correct.

        One of the things reported on in the book is an experiment using different wordings of the problem.  No matter how clearly the problem was stated, very few people got it right.  (Although, when it was unclearly stated, even fewer got it right).

        •  People get it wrong because we tend to (2+ / 0-)
          Recommended by:
          plf515, NCrissieB

          greatly overestimate our ability to reason correctly about probability without training or practice in the subject.  E.g., consider the huge number of people who throw their money away by playing the lotteries, which are essentially scams that take advantage of the average person's confusions about probability.  

          Or consider the widespread belief in the "hot hand" in basketball, or the belief that you are more likely to get tails if you've flipped more heads previously, etc.

          Many people also tend to think that there is some "absolute" probability of an event (outside the quantum realm), independent of one's state of knowledge.  But probability is a function of both an objective state of affairs and one's state of knowledge about it, so if one of the arguments of the function changes, the value of the probability will also change.

          •  Hot hand (2+ / 0-)
            Recommended by:
            plf515, NCrissieB

            Why throw the "hot hand" example in there? Has there been some kind of study? It's not simple probability, since it's human beings who can, for example, psyche themselves out or play with extra confidence.

            Deranged neoconservative militarism isn't the solution to nuclear proliferation; it's a cause. -- Glenn Greenwald

            by factbased on Tue May 05, 2009 at 09:02:18 AM PDT

            [ Parent ]

            •  There have been studies, yes. (1+ / 0-)
              Recommended by:
              plf515

              The player's last N shot results are no predictor of whether the player will make the next shot.  There is no statistical correlation.  There may be predictors in the player's body (confidence, anxiety, good form, injury, etc.), but the last N shot results are just statistical noise.

              Good morning! ::hugggggggs::

              •  Got a link? (1+ / 0-)
                Recommended by:
                plf515

                I'd like to see the studies. I'm fairly certain that we'd clearly see the hot hand phenomenon in a controlled experiment (e.g., shooting 100 uncontested 3-point shots in a row at noon every day for a month). If it doesn't show up in game data, I can think of a few factors that may mask it. For example, a hot shooter can be double-teamed. A fatigued player is rested. An anxious player passes the ball for a while and only takes high-percentage shots.

                How about an Office of Fact Based Initiatives?

                by factbased on Wed May 06, 2009 at 03:53:55 PM PDT

                [ Parent ]

          •  If you've flipped enough heads in a row ... (2+ / 0-)
            Recommended by:
            plf515, guyeda

            ... you should begin to question whether: (a) this is a two-headed coin; or, (b) the way you're flipping the coin is skewing the outcome toward heads.

            Which is to say, it's totally counter-intuitive.  After 20 consecutive heads outcomes, the "law of averages" doesn't say sooner or later you're due for a run of tails.  Given that many heads outcomes in a row, the statistics suggest this isn't the 50/50 situation you thought it was.

            As to whether a player gets a "hot hand," you can't assess that by the last N shots taken.  Players can shoot better or worse than their season average would suggest, but the cues for whether and why they are doing so are in their bodies - signs of confidence, anxiety, good form, injury, etc. - rather than the last N shot results.  The last N shot results are just statistical noise, except to the extent that they affect the player's body (confidence, anxiety, etc.).

            Good morning! ::huggggggggs::

            •  I still say you're wrong on this (2+ / 0-)
              Recommended by:
              guyeda, NCrissieB

              but there's probably no point in our getting into it again.

              •  On the coin toss or the "hot hand?" (1+ / 0-)
                Recommended by:
                plf515

                If you toss a coin and get 20 heads in a row, the probability of that happening by random chance is 0.5^20 or about 10^-6.  It's far more likely that it's either a two-headed coin, or you're not giving it a fair toss.

                As to the "hot hand," the study you've shown was only based on the last N shots.  I agree the results of the last N shots are statistical noise and don't say anything about the next shot.  But the body cues (confidence, anxiety, good shooting form, injury, fatigue, etc.) do, and they exist independent of the results of the last N shots.  The player can have made the last N shots but just sprained an ankle, or missed the last N shots but you can see the he's confident, relaxed, and using good form.

                If he made the last N shots but just sprained an ankle, you replace him with a healthy player.  If he missed the last N shots but everything else about his body says he's fine, you leave him in because given time (though perhaps not in this game!) he'll make his percentage of shots ... unless he gets frustrated about having missed several in a row.  Then maybe you rest him to let him get out of that anxiety.

                •  On the coin toss I agree with you (1+ / 0-)
                  Recommended by:
                  NCrissieB

                  If you  get 20 heads (or 10, or even 5) in a row, you start to get suspicious.

                  On the shooting - I guess we are disagreeing as to what a streak is.  If the previous N shots tell you nothing about the next shot, to me, that proves that streaks don't happen.  

                  •  Streaks are statistical noise. (1+ / 0-)
                    Recommended by:
                    plf515

                    If a 50% field goal shooter in basketball makes four shots in a row, that tells me nothing.  That's going to about once in any 32 series of four shots ... and every shot he takes can be treated as starting a new series.  So assume he's one of his team's scorers, so he probably averages about 18 shots a game.  That's an average 15 chances per game to go on a "streak" of four made shots in a row.  (If he takes 18 shots a game, shots 16, 17, and 18 can't start a four-in-a-row "streak" as he won't have enough shots left to finish it.  Only his first 15 shots can start a "streak.")  The probability he'll have such a "streak" by random noise is:

                    p = 1-((31/32)^15) = 0.379

                    or about 38%.

                    Explanation for non-math-geeks: the probability of his making four shots in a row, given a 1/2 chance of each, is 1/2 x 1/2 x 1/2 x 1/2 = 1/32.  If the probability that a given shot will start a four-in-a-row "streak" is 1/32, then the probability that a given shot will not begin such a "streak" is 31/32.

                    The probability that none of his first 15 shots will begin such a streak is (31/32)^15 or 0.621.  Subtract from 1, and you have the probability that at least one shot will start such a "streak," or about 38%.

                    Note to math-geeks: I'm pretty confident on the math here, but do let me know if I've made a mistake, please?

                    That's a high enough percentage that it shouldn't be surprising if he has a four-in-a-row "streak."  You should expect it by by statistical noise - the ordinary clumping of random events - just as given 18 coin tosses you have a 38% chance of at least one streak of four "heads" outcomes.

                    So yes, "streaks" are statistical noise.

                    But good play isn't.  Good play is a function of training, experience, fundamentals, strategy and decision making, confidence or anxiety, fatigue, and injury.  A player's training and experience are reasonably stable for a given season, but the rest can and will vary game-to-game, and within a game, because he/she is human.

                    Note: As a coach I focused on decision making and fatigue.  Decision making and confidence/anxiety are interrelated.  You can usually see confidence or anxiety in a player's face and body, but it will also be evident in his/her decisions.  Fatigue degrades decision making and fundamentals.  A player with tired legs will make mistakes because he/she can't get to the right spot in time, and his/her legs aren't supplying enough power for a proper shooting form.

                    In terms of shooting, a player who's "playing well" is using good shooting form (fundamentals), working for and taking good shots (strategy and decision making), is confident rather than anxious, has fresh legs, and is healthy.  But his season-long statistics include times when one or more of those conditions is not true: he/she is using bad form, forcing bad shots, anxious rather than confident, fatigued, and/or injured.  In fact, his/her season-long statistics are skewed toward periods when he/she is not "playing well," because it's less likely that all of those conditions are true at any given moment.  So when he/she is "playing well" - when all of the positive conditions are true - you can expect him/her to "shoot better than his/his stats."

                    But the key is this: you can't assess those conditions by looking at his/her last N shot attempts (results).  There will still a lot of statistical noise mixed into the results.  To assess those conditions, you have to look at how he/she is playing (process).

                    That's why you can win that bar bet on whether the NBA 50% shooter who's made his last four shots will hit the next one, while a good coach knows to rest a player because his game is off.  The fan you're winning money from in the bar is looking at results.  The good coach is looking at process.

                    •  I disagree (1+ / 0-)
                      Recommended by:
                      plf515

                      I've thought about this some more and I disagree with you. Are these fair representations of your 2 claims above?

                      1 - performance in the recent past does not affect current performance
                      2 - fatigue, anxiety and injury affect performance

                      I don't see how those can both be true unless each of the performance affecting factors is only an instantaneous condition. But I think fatigue (get enough sleep last night?) and injury (hamstring a little sore?) can certainly last all game.

                      I think that good conditions (lots of sleep, confident, injury-free) could make a 50% shooter a 55% shooter for the night. Similarly, bad conditions (couldn't sleep, anxious, sprained finger) can make a 50% shooter a 45% shooter for the night. Either one could have a 4 shot streak, but the former is more likely to have it than the latter. For example, the likelihood of starting the game with a 4 basket streak is ~9% versus ~4%.

                      How about an Office of Fact Based Initiatives?

                      by factbased on Wed May 06, 2009 at 03:41:48 PM PDT

                      [ Parent ]

                      •  I don't know about Chrissie's (0+ / 0-)

                        2nd point, I just don't have data.

                        But there is a lot of data, at least for basketball, showing that there is no 'hot hand' - a player's chance of hitting a shot after, e.g. making 4 in a row is no different than the same player's chance of hitting after missing 4 in a row.

                      •  Fatigue varies during the game. (1+ / 0-)
                        Recommended by:
                        plf515

                        It's not about whether you got enough sleep last night.  It's about how a player's energy levels change over the course of a game.  A starting player runs about 8 miles in the course of an NBA game.  In a college game it's about 6 miles.  And unlike a long-distance runner, that running comes in sprints and stops.  It's exhausting, and that's why coaches rest players during the game.

                        Fatigue levels definitely vary during the game.  It's not fixed for the duration of a given night.

                        •  Yes, fatigue varies (1+ / 0-)
                          Recommended by:
                          plf515

                          But unless you have evidence that lack of sleep does not affect performance, I'll stand by that point. And the others I made.

                          I do believe that people greatly overestimate winning streaks. I also believe there are performance affecting factors at the beginning of a game that can be substantially in effect the entire game.

                          How about an Office of Fact Based Initiatives?

                          by factbased on Fri May 08, 2009 at 04:53:10 PM PDT

                          [ Parent ]

                          •  Argument by equivocation. (0+ / 0-)

                            I defined fatigue and injury in terms that vary within a game.  You changed their definitions to be constant throughout the game.  That's argument by equivocation.

                            Yes, there are performance factors that don't change much during a game.  I included two in my original response: training and experience.  There are also fatigue and injury issues that are fairly constant throughout a game (e.g.: stomach virus that kept a player from sleeping), or an entire season or career (e.g.: chronic, irreparable joint injury).

                            But there are also performance factors that change within a game.  Fatigue as I defined it (fresh vs. winded) and minor injuries (e.g.: muscle cramp) are among those.  They won't always show in the last N shot attempts, because you shouldn't expect much regression to the mean (the Strong Law of Large Numbers) in a sample of only 3 or 4.

                            A coach may recognize those factors and rest a player whose last N shot attempts are at or above a season norm, because the coach sees that the player isn't playing well and those last N shots are just noise.  Indeed that happens quite often, as most coaches have planned substitution rotations over the course of a game that take advantage of known game breaks (e.g.: end of 1st or 3rd quarter), to get an "extra" minute or two rest for a player when the action is stopped.

                            As to winning streaks, I agree with you.  There are sometimes reasons a team wins or loses several games in a row - e.g.: playing a series of weak opponents, or having key players out with injury - but in most cases it's just statistical noise.

                          •  When you started this thread (0+ / 0-)

                            you didn't define fatigue and injury in terms that vary within a game. Maybe I missed it elsewhere. But once you narrowed the definition of fatigue, I agreed and stood by my point about sleep or lack of sleep without using the word fatigue.

                            You seem to agree that there are game-length performance-affecting factors. So was it just semantics you disagreed with? Note that I find the existence of short-lived factors like being fresh or winded to be obvious and uncontroversial.

                            How about an Office of Fact Based Initiatives?

                            by factbased on Mon May 11, 2009 at 11:23:19 AM PDT

                            [ Parent ]

  •  I once made a fool of myself at the whiteboard .. (5+ / 0-)

    I was seventeen.  I was good enough and lucky enough to get a week at a little place called Villiers Park where twenty of us maths prodigies were given some intense instruction.  One of the seminars was around probability and the Monty Hall problem came up.  I was convinced that after opening the door, the probabilities changed to 50-50.

    So I went up to the whiteboard and used Bayesian probabilities (p(A v B) = p(A) * p(B | A) = p(B) * p(A | B)) to prove that I was completely wrong.  "Huh", I said, "I'm convinced." and went and sat down.

  •  I think the best intuitive explanation... (4+ / 0-)
    Recommended by:
    Orinoco, plf515, winterbanyan, NCrissieB

    Is that you are using the rules of the game to change the goal from choosing the door with the car (33%) to choosing a door with a goat (66%). this seems to make a lot of sense to people.

  •  The way to win at the game (9+ / 0-)

    is if you're Amish; then you'll want the goat and the odds are in your favor. :-)

  •  Our brains (9+ / 0-)

    are pattern recognition machines, not probability engines. Intuition can easily be fooled.

    For example, Mary was a hippie who grew up in a commune, had numerous body piercings, and changed her hair color weekly. After twenty years, which do you think is more likely?

    1. Mary paints her toenails and works in a bank.
    1. Mary works in a bank.

    Surprisingly, many people pick choice 1 even though choice 2 is at the very worst at least as likely as choice 1. Mathematically, the probability of A is always greater than or equal to the probability of A and B.

    If you don't know where you're going, any road will do.

    by exregis on Tue May 05, 2009 at 06:07:26 AM PDT

    •  Sort of.... (4+ / 0-)
      Recommended by:
      SoCalJayhawk, Dave in RI, plf515, Toon

      We're actually better at probabilistic reasoning than many mathematicians care to admit.  Problems like the Monty Hall issue posit something that usually doesn't exist in the real world: a finite, knowable sample set.  In the real world, the sample set may (or may not) be finite, but it's often far too large to be even computationally knowable.

      So we fall back on other tools which are less precise, but which can still be rigorous and very useful in a probabilistic way.  They don't guarantee success, but they "nudge the odds" toward success.  And we humans have been "nudging the odds" toward success well enough, often enough, to have survived and thrived very well as a species.

      A classic example are studies that show we tend to "over value" short-term risks and rewards and "under value" long-term risks/rewards.

      It's true that we weigh short-term risks/rewards more heavily than long-term risks/rewards, but does that mean we "over value" them?  There is, after all, a non-zero probability that we won't be alive to see a long-term risk/reward play out, and the longer the term the higher that probability.  Even if we don't die, the longer the term, the greater the probability that intervening causal elements and/or later decision points will change the risk:reward calculation in ways we can't precisely estimate.  Given those vagueries, we tend to weigh long-term risk/rewards lower than a mathematical calculation might suggest.

      Does that mean we're reasoning poorly ... or that the mathematical calculation isn't considering enough of the real-world variables that we weigh, often without being able to articulate them?

      Good morning! ::hugggggggggs::

  •  simplest explanation (5+ / 0-)
    If you don't switch, you win if your door has the car: 1/3 of the time.

    If you do switch, you win if your original door had a goat: 2/3 of the time.

    Doesn't have to be more complicated than than. Of course the reason is that Monty is giving you information. If you picked a goat at first (which is what you want!) Monty will then tell you where the other goat is, and therefore where the car is.

  •  Great fun. Thank you plf515 :D (2+ / 0-)
    Recommended by:
    plf515, NCrissieB

    The Shape Of Things "Beware the terrible simplifiers" Jacob Burckhardt, Historian

    by notquitedelilah on Tue May 05, 2009 at 06:14:14 AM PDT

  •  Car Talk Puzzler (4+ / 0-)

    A few years ago, this problem (complete with car and goats) was used as a puzzler on car talk on NPR.  IIRC, they got it completely right.

  •  Here's a problem ... (4+ / 0-)
    Recommended by:
    plf515, Justus, guyeda, NCrissieB

    ... not exactly a probability problem but with some similarities to the Monte Hall poser. It comes from a recent article in the Economist about brain functioning. (plf --This might also be of interest in the GEB series.) Experimenters did brain scans on subjects who were working on problems. The experiments found that a section of the brain associated with insight would fire up seconds (I think as many as 10 seconds; can't remember) before the subjects became aware they'd come up with the answer. In other words, they'd solved the problems subconsciously. Anyway, to do the experiments, the experimenters had to have problems that were challenging enough to require a bit of thought but easy enough so that at least some of the subjects could solve them. This is the example the article used:

    You're on the first floor of a building and have three switches, one of which operates a light bulb on the second floor. You can make one trip to the second floor to check the light's status. How can you be sure of finding the switch to turn the light on?

    •  Leads to key test-taking (and life) strategy: (4+ / 0-)
      Recommended by:
      Dave in RI, plf515, elropsych, kktlaw

      If you "feel stuck" after considering a problem for a reasonable time, set it aside and do something else if you can.

      When taking a test, skip it for now and solve the other problems.  Doing so may well help pieces fall together that help you come back to solve the problem you got "stuck" on.  At the very least, you won't waste valuable test time being "stuck."

      In life generally, the same principle often applies.  That sense of "feeling stuck" is the absence of what you described from that article, the intuitive sense of "I know I can figure out the answer here" because at some level your invisible reasoning already has.  Pressing on with a problem when you "feel stuck" is usually an exercise in mounting frustration.  Better to do something else - if you can - and come back to that problem later.

      Obviously sometimes you can't set the problem aside, and in that case you should make the most cautious choice you can for now and hope you get a chance to revisit it once you know more.

      Good morning! ::hugggggggggggs::

      •  Happpens a lot with crossword puzzles... (2+ / 0-)
        Recommended by:
        plf515, NCrissieB

        ... Get stuck, walk away for a while (maybe even a day), return with one or two answers immediately upon picking up the paper again. Happens almost every time.

      •  There's a name for this interval... (4+ / 0-)
        Recommended by:
        Dave in RI, plf515, NCrissieB, kktlaw

        incubation period. (source at end of comment, for the curious)

        The hippocampus processes both memory recall/recognition and stress responses in the limbic system. An odd combination on the surface, but for evolutionary reasons there is some sense to the arrangement.

        If the stress level rises enough, the hippocampus can flip from working on the memory to marshaling the brain's and body's response(s) to the stress.

        Test anxiety can be small, simple, and subtle but debilitating as a result.

        Allowing oneself the right to be uncertain, to take some time to think about something else or some other problems can re-cue the hippocampus to focus on a new memory problem and flip back, literally dropping the stress response.

        Manipulation of incubation periods and the meta-cognition behind that activity are skills I spend a lot of time with in the classes I teach.

        EEK = Even Emotional Keel

        I like the irony.

        On other occasions, however, this cognitive effort proves fruitless and the correct solution eludes the thinker. In these cases, Wallas argued, thinkers enter an incubation stage in which they no longer consciously thinks about the problem. Wallas (1926) actually distinguished between two forms of incubation: "the period of abstention may be spent either in conscious mental work on other problems, or in a relaxation from all conscious mental work" (p. 86). Wallas believed that there might be certain economies of thought achieved by leaving certain problems unfinished while working on others, but he also believed that solutions achieved by this approach suffered in depth and richness. In many cases of difficult and complex creative thought, he believed, deeper and richer solutions could be achieved by a suspension of conscious thought altogether, permitting "the free working of the unconscious or partially conscious processes of the mind" (p. 87).1 In either case, Wallas noted that the incubation period was often followed by the illumination stage, the "flash" (p. 93) in which the answer appears in the consciousness of the thinker. (This answer, too, is subject to verification.)

        Wallas (1926) was quite certain that incubation involved "subconscious thought" (p. 87), and that the "instantaneous and unexpected" (p. 93) flash of illumination reflected the emergence of a previously unconscious thought into phenomenal awareness

        Rogues are preferable to imbeciles because they sometimes take a rest. - Alexandre Dumas

        by elropsych on Tue May 05, 2009 at 10:14:09 AM PDT

        [ Parent ]

    •  Ok. Ready for solution? (3+ / 0-)
      Recommended by:
      plf515, guyeda, NCrissieB

      I needed help and so figure some of you do, too.

      Think heat.

      Solution in reply.

  •  Poll question too easy (3+ / 0-)
    Recommended by:
    plf515, Justus, NCrissieB

    Obviously the writing mathematicians are a self-selecting sample. This sampling of US mathematicians is about as valid as using a Daily KOS poll to sample average Americans.

    The average American is obviously not as well informed, thoughtful, or good looking as the average Kossack.

    Smiting trolls on the tubes since 1977!

    by blue aardvark on Tue May 05, 2009 at 06:29:49 AM PDT

  •  SPOILER to the poll question (3+ / 0-)
    Recommended by:
    northsylvania, elropsych, NCrissieB

    the answer is

    You do not have enough information.

    To be able to tell what percentage of mathematicians get this wrong, you  would have to randomly sample mathematicians.  The mathematicians who write in are not at all a random sample.  I would guess that people are much more likely to write to vos Savant in order to disagree than to agree.  

    Suppose, for example, that 2000 mathematicians read her column, and that 500 disagreed with her solution, while 1500 agreed.  But 450 of the 500 wrote in to tell her she was wrong, but only 50 of the 1500 wrote in to tell her she was right.

    This is similar to the use of "complaint cards" - e.g., those cards in hotel rooms that say something like "we want to know what you think".  Hotels do NOT use these to see how well they are doing, and they are right not to do so.  They use them to find particular problems and to reward particularly good employees.  Because people are much more likely to fill them out if something is awful or fantastic than if everything is OK.

  •  the 100 Door thing explains it best (5+ / 0-)

    I used to do the Monty Hall Problem as 10 door, where Monty opens 8
    after your choice.

    Suddenly it became a lot clearer to people.

    I view it as Set theory problem not a probability problem.

    That you have a probability of 33% of getting a set with a car on the first try
    and there is a 66% probability of not getting into a set with a goat and a car.

    Once Monty gives you some information, you still know that bigger probability is
    condensed around the remaining door.

    If you do the problem as 10 doors or a 100 doors it's pretty easy for most people.

    you have a 10% chance of getting the car the first time, and a 90% of missing the set with 8 goats and a car.    If you switch,  90% of the time the car is most likely behind the remaining car.

    George Bush is Living proof of the axiom "Never send a boy to do a man's job" E -2.25 S -4.10

    by nathguy on Tue May 05, 2009 at 06:35:04 AM PDT

    •  There is a very simple solution to the Monty Hall (1+ / 0-)
      Recommended by:
      nathguy

      problem, which is given as follows:

      Take the 10-door version of the MH problem. If all the doors are closed, your chosen door will have a 1/10 chance of being correct. The remaining 9 doors "collectively" will have a 9/10 of having the correct answer. If the host now opens 8 of the remaining 9 doors empty, then the remaining unchosen door will still retain a 90% of being correct, for two reasons,:

      (a) the host knew the correct answer and withheld the answer, or  

      (b) even if the host did not know the correct answer, by blind luck he did not choose the correct door...

      In either case, there is a 90% probability of choosing the correct door by switching from the original choice, which still remains at 10%.

      Thus, the 3-door Monty Hall problem is the simplest form of the 10, or 100, or 10,000 door problem. Remember, computer solutions of the 3-door MH problem are not programmed to know the correct answer. In other words, the original 1/3 odds for the original chosen door does not change, while there is a "collective" 2/3 chance for the two remaining doors to have the correct answer. By switching doors in the 3-door MH scenario, one simply shifts from the "one door 1/3 odds set" over to the "collective two door 2/3 odds set" (again, assuming that the door that is opened by the host is empty).  

      The key to understanding the Monty Hall problem is the concept of "group statistics", which current probability theory does not allow for. Only by using elementary set theory in probability statistics will this logical confusion among professional mathematicians get finally fixed. This is the main reason most statisticians (or pure number theorists, such as Paul Erdos) do not get (or even understand) the correct answer to the Monty Hall problem...

      "The blackbird whirled in the autumn winds. It was a small part of the pantomime." Wallace Stevens

      by mobiusein on Tue May 05, 2009 at 07:28:54 AM PDT

      [ Parent ]

      •  This is incorrect (1+ / 0-)
        Recommended by:
        Toon

        It makes a great deal of difference whether Monty knows which door has the car, and the computer programs WERE programmed to know that.

        If Monty does not know which door, then switching makes no difference

        If Monty does know which door, then switching improves your odds.

        •  the monty hall problem defines (1+ / 0-)
          Recommended by:
          plf515

          monty as knowing which door to open to reveal a goat.

          now you could define it as

          "Monty asks Jay to please open a door,"  but
          it doesn't matter what monty knows, merely that the act of opening
          a door, compresses the other door set to have a much higher probability
          of being a car..

          George Bush is Living proof of the axiom "Never send a boy to do a man's job" E -2.25 S -4.10

          by nathguy on Wed May 06, 2009 at 06:48:07 AM PDT

          [ Parent ]

          •  It does matter (1+ / 0-)
            Recommended by:
            plf515

            In the puzzle statement, Monty knows and always opens a goat door, telling you nothing about your own door.

            pick the car: 1/3
             - monty opens a door with a goat: 1/3
               - stay to win or change to lose
            pick a goat: 2/3
             - monty opens a door with a goat: 2/3
               - stay to lose or change to win

            Results: Stay wins 1 in 3. Change wins 2 in 3.

            If Monty didn't know and opened a non-chosen door at random, you get a goat revealed 2/3 of the time and a car 1/3 of the time. If you still get to change after that, you get a no-risk win 1/3 of the time.

            pick the car: 1/3
             - randomly open a door with a goat: 1/3
               - stay to win or change to lose
            pick a goat: 2/3
             - randomly open a door with a goat: 1/3
               - stay to lose or change to win
             - randomly open the door with the car: 1/3
               - change to win

            Results: No risk win 1 in 3. In the uncertain choice cases (2 in 3), stay wins 50% and change wins 50%.

            Deranged neoconservative militarism isn't the solution to nuclear proliferation; it's a cause. -- Glenn Greenwald

            by factbased on Wed May 06, 2009 at 09:27:20 AM PDT

            [ Parent ]

      •  It's stunning Paul Erdos gets it wrong (1+ / 0-)
        Recommended by:
        plf515

        I studied information theory as part of my undergrad, so,
        I understand how a little bit of information wildly varies outcomes for probabilistic events,  and that conveyed information should alter gaming strategies.  

        I have sat down with lots of smart guys and been the only one who has it
        right on the monty hall problem, so it is very counter intuitive.

        George Bush is Living proof of the axiom "Never send a boy to do a man's job" E -2.25 S -4.10

        by nathguy on Tue May 05, 2009 at 07:41:48 AM PDT

        [ Parent ]

      •  Not quite.... (1+ / 0-)
        Recommended by:
        plf515

        If there are 100 doors and the computer opens them totally at random, there's a very good chance the computer will open the "right" door long before you're down to only two doors.  Among the times when the computer hasn't already opened the right door, once it gets down to two doors, it's a 50/50 question as to which door is the winner.  The other probabilities are included in the times you never get to a two-door problem because the computer shows you the winning door earlier.

  •  That update really gets to the core of it all (4+ / 0-)
    Recommended by:
    Dave in RI, snazzzybird, plf515, NCrissieB

    Once Monty opens a door, two things happen. First, the situation ceases to be random because an intentional act that limits possibilities of a certain type is introduced. Second, the calculation moves from the odds of a single event happening to the odds of a series of events happening. The key to getting people to understand the problem is getting them to understand these two things so that they aren't comparing apples to oranges which is the way that the problem usually gets staged in our heads.

  •  in defense of mathematicians (4+ / 0-)
    Recommended by:
    Dave in RI, plf515, AnnArborDem, NCrissieB

    speaking as a former pure mathematician married to an applied mathematician, i know it is perfectly understandable why so many mathematicians get this problem wrong.

    the key, i think, is not recognizing the impact of the change in information after the second door is opened. statisticians may be more comfortable at dealing with dynamic dependent and independent variables, but most mathematicians (and presumably the public at large) are accustomed to problems without changing conditions. we expect the location of the car to be truly random, but monty changes that.

    freedom isn't free, but it isn't dumb either.

    by astro on Tue May 05, 2009 at 06:52:12 AM PDT

    •  I agree half-way. :) (1+ / 0-)
      Recommended by:
      plf515

      I think you've probably nailed the reason so many mathematicians get it wrong.  The reason so many people in general get it wrong may differ, however.

      I think most people are very accustomed to problems with changing conditions, as that's called "life."  And a lot of the time, we discover the right answer is to change our minds when we've learned something new.  ("I'll drive to work on the freeway.  Oops, there's an accident on the freeway; I'll take the surface streets.")

      What we often don't recognize in the Monty Hall Problem - but often do recognize in the vagueries of real life experience - is how new information changes the decision matrix.

      What I find curious is why we don't recognize that in the Monty Hall problem ... but do get off the freeway (or don't get on it) when we hear there's been an accident....

      Good morning! ::huggggggggs::

  •  The real problem in understanding this (3+ / 0-)
    Recommended by:
    plf515, winterbanyan, NCrissieB

    is the problem of distinguishing probabilities from facts....I think the real cognitive dissonance comes from the idea that one of two unknowns becomes more likely while neither is actually known. The fact that switching improves the odds does not mean that it guarantees a win, that's the problem. Our minds are designed to deal with two kinds of information--that which is known, and that which is inferred based on other knowledge--when you get into abstract probabilities human "intuition" is worse than useless. That's the basis of the casino business.

    On the other end, when we are faced with statistical probabilities that are almost certain--like when there is less than one percent likelihood of something, we tend to assume that thing is impossible.

    "All governments lie, but disaster lies in wait for countries whose officials smoke the same hashish they give out." --I.F. Stone

    by Alice in Florida on Tue May 05, 2009 at 07:04:30 AM PDT

  •  kinda spooky (3+ / 0-)
    Recommended by:
    plf515, winterbanyan, NCrissieB

    Just before bed last night, I started watching the movie 21 on my Netflix player...

    Didn't get very far (it was late), but got far enough to see Kevin Spacey present that problem to his genius students.

    Went to bed thinking about it... Had fallen into the common 50% trap and was reasoning my way out of it (My doctoral minor was statistics, so I expected better of myself, but it was late and I was tired, and, well, I got suckered.)

    Anyway, I dreamt about the problem, and probability, and all sorts of things I haven't thought about since college, and here this morning when I woke up is the answer with a beautiful, simple explanation.

    Thanks for the wake-up call!!! But what were you doing listening in on my dreams?!?!?

  •  NY Time flash game version (2+ / 0-)
    Recommended by:
    plf515, winterbanyan

    The NY Times has a fun time-wasting flash game for this is you want to test it empirically.  You can easily run the simulation either way 50 times.  

    Also, I humbly point out an old statistician joke of mine from the archives (I know, I know, it's ageist to make fun of old statisticians).

    "There is more stupidity than hydrogen in the universe, and it has a longer shelf life." Frank Zappa

    by zootfloggin on Tue May 05, 2009 at 07:08:39 AM PDT

  •  It's simple really. (5+ / 0-)

    When you make your first door pick, you divide the doors into two groups: "your door" and "Monty's doors".

    Behind Monty's doors are either a goat and a car, or two goats. In the full knowledge of what is behind those two doors, Monty eliminated one goat.

    That's exactly what you would do, so suppose you were given the equivalent choice, instead of Monty doing it for you. You can either pick your one door, or you can open both of Monty's doors and then eliminate one of them, in the full knowledge of what's behind them.

    Obviously you will eliminate a goat, just as Monty did. You're not going to eliminate the car if it's behind one of Monty's doors.

    So even though this is an exactly equivalent situation (i.e. you will make the same elimination Monty did) the choice is clearer: you are being asked to choose between one door ("your door") or two doors ("Monty's doors"). Your odds are twice as good with two doors as with one. See?

    One nation, indivisible.

    by Doctor Frog on Tue May 05, 2009 at 07:09:27 AM PDT

  •  Huh? (1+ / 0-)
    Recommended by:
    plf515

    I will be the first to admit that I'm not the sharpest knife in the drawer, so please go slowly.

    One-third of the time the car is behind your door; two-thirds it's behind the other two.  Let's call your door A, so Monty has B and C to work with.  So you conduct the first sampling, in which your door A has a one-third chance.

    But two-thirds of the time you don't have the car, and Monty simply opens whichever of the other two doors doesn't have the car, so two-thirds of the time the unopened door holds the car.

    Pardon?  There's a two-thirds chance of the car being behind B and C combined, not behind one or the other.  There was always a 100% probability that there was a goat behind either B or C, so by showing a goat behind door C, he has revealed nothing to you.  All he has done is reduce the probability of a car behind door C to zero.

    Now he asks you to make a second sampling, between A and B.  Two doors, one has a car.  That looks 50-50 from here.  Same as if there were three cards face down on the table, two kings and a queen.  Object is to pick the queen.  You pick one, and the dealer (who knows which ones are which) turns a king and removes it.  Two cards left, 50-50.  The odds change as the sample changes.  Just as your odds of being dealt a particular card change as the deck is dealt.

  •  Mathematically, this is all true. (3+ / 0-)
    Recommended by:
    TrueBlueMajority, plf515, NCrissieB

    However, I used to watch Monty on Let's Make a Deal as a kid and I know from watching that he didn't always give folks a choice like this and he often only gave people the choice when they had hit the 33.3% jackpot.  Remember, he not only knows where the car is, but he can offer a new deal or not, based on his whim alone.  His offer of this new deal by revealing a door and offering you a chance to change your picks is neither guaranteed to occur every time or even random. He does it when he wants based on his reading of you and his desire to add spice to the program.  What if, by analyzing Monty's behavior over a month of shows, you determined that 2/3 of the times he offered the new deal, it was because the lucky contestant had picked the car the first time.  How would this affect the math? Hasn't he negated the advantage of switching?

  •  This is still and will always be bullshit (0+ / 0-)

    Monty opens a goat door no matter what, so you're really only choosing between two doors. He always opens a goat door, so you can basically ignore what Monty does as part of the probability. The true choice is between your door and the car door, the extra door is there just to confuse people and make for better television.

    The real probability only kicks in after the first round, when you have two options, and thus your initial selection is 50/50. The mathmeticians get all breathless and excited by all the other shit and miss the fact that their numbers are wrong because they're not looking at the problem right.

    •  Wrong (3+ / 0-)
      Recommended by:
      TrueBlueMajority, factbased, Toon

      You are wrong, you have been proven wrong by countless simulations, as well as by formal proof.

      it is not 50/50, and you are the one looking at the problem wrong.

      Just because you are choosing between two doors does not mean that they are equally likely.

    •  If Monty offers the choice on a purely random (4+ / 0-)
      Recommended by:
      plf515, factbased, NCrissieB, Toon

      basis and not because you chose the right one the first time, (which in the real world, he could do) he is, in effect, allowing you to choose the two doors you did not choose the first time by allowing you to change picks.  It is more likely you chose the wrong door.  1/3 chance right, 2/3 chance wrong.  By revealing a door you didn't pick and allowing you to select the other door you didn't pick, he is giving you two doors as opposed to your initial one. The only thing I'd want to know (from analysis of his past behavior) is whether Monty is more likely to offer this choice when he knows you've chosen the right door the first time. If he does have a bias in his offer, it throws the math off.

  •  Why I love my wife. (5+ / 0-)
    Recommended by:
    plf515, Tomsank, guyeda, seenaymah, NCrissieB

    Weeks ago my wife and I were talking about this problem.  I explained all the probability as my brilliant son had explained to me.  She still wants to keep her door.  She explains that in a larger sense, Maybe she is supposed to have that goat.

    I kissed her.

  •  My explanation (4+ / 0-)
    Recommended by:
    plf515, guyeda, NCrissieB, Toon

    goes like this:

    First, you divide the doors into Chosen and Unchosen groups.

    Second, list what you know about each group. The Chosen group has 1 door which has a 1/3 chance of having the car and a 2/3 chance of having a goat. The Unchosen group has 2 doors. As a group, it has a 100% chance of having a goat, a 2/3 chance of having the car and a 1/3 chance of having a second goat.

    Third, understand that the group odds above remain until you get new information about a group as a whole.

    Fourth, when Monty opens a door, he always opens a door with a goat in the Unchosen group. This is not new information about the Unchosen group since you already knew the group had a 100% chance of having a goat. The group still has a 2/3 chance of having the car and a 1/3 chance of having a second goat. The only new information Monty has provided is internal to the Unchosen group, letting you claim the full 2/3 odds of getting a car by changing your pick to the single unopened door of that group.

    Deranged neoconservative militarism isn't the solution to nuclear proliferation; it's a cause. -- Glenn Greenwald

    by factbased on Tue May 05, 2009 at 08:07:04 AM PDT

    •  Nice. I tried that tack above, but I think (3+ / 0-)
      Recommended by:
      plf515, factbased, NCrissieB

      your explanation is better. That works assuming Monty always offers the new deal.  What if he doesn't always offer a new deal? What if from an analysis of Monty's past behavior you determine that about 2/3rds of the time, he offers the new deal only when the chooser has chosen the car the with his first choice.  How does this affect the thinking?  Has the probability now collapsed to a 50/50 choice?

      •  Thanks (1+ / 0-)
        Recommended by:
        NCrissieB

        I've explained it several ways over the years and that's the one that makes it most obvious to me. Sometimes I do the 100 doors. Or just say pick a number between 1 and a million and see if it matches the number I write down. Then narrow it down to 2 numbers. Do you want to stay with your number or switch to 247117?

        I replied to your post further up. I think I followed the new rule you used up there, but I'm not sure I understand the way you state it here.

        Deranged neoconservative militarism isn't the solution to nuclear proliferation; it's a cause. -- Glenn Greenwald

        by factbased on Tue May 05, 2009 at 09:38:38 AM PDT

        [ Parent ]

  •  Three items: Reformulation, Bayes & Prisoners! (2+ / 0-)
    Recommended by:
    plf515, NCrissieB

    The easiest way to explain the Monte Hall problem is to slightly reformulate the problem.  As it stands causality appears to be important (and thus the time sequence of events), but this is not the case.  So, consider the following equivalent formulation (and indeed the one you would code up!):

    You can either choose:
    A. 1 door
    B. 2 doors and I'll remove the wrong choice.

    Here it is obvious which is better.  This is, of course, the 100 door example.

    The nice bit of the problem is related to Bayes theorem!  That is, it illustrates how new knowledge updates probabilities.  In this case the new knowledge includes both the goat and the fact that Monte Hall isn't going to show you the car.  This is precisely where intuition breaks down, and most "explanations" fail because they've chosen some scheme for performing the update without any real justification.  For this reason, this is usually a jumping off point for Baysian theory, and yet none here!  Why?

    Finally, an associated problem, which is more applicable to this crowd is the Three Prisoners Problem.  It is a cousin of the Monte Hall problem, and proves that curiosity kills!

    There are three prisoners scheduled to be executed, A, B, and C, although one will be pardoned. A asks the warden to tell him the name of one of the others who will be executed. As the question is not directly about A's fate, the warden obliges—secretly flipping a coin to decide which name to give A if A is the one being pardoned. Assuming the warden's truthfulness, there are now only two possibilities for who will be pardoned: A, and whichever of B or C the warden did not name. Did A gain any information as to his own fate, that is, does he change his estimate of the chances he will be pardoned?

    The answer is, yes!  He now has doubled his chance of being executed!  That is, unlike on Let's Make a Deal, prisoner A can't "switch doors", i.e., become prisoner B or C.

    Justice deferred is justice denied. -MLK

    by zephron on Tue May 05, 2009 at 08:10:52 AM PDT

    •  Hrmm.... (0+ / 0-)

      1 - Ax, "Bx," C! (p=1/3)
      2 - Ax, B!, "Cx" (p=1/3)
      3.1 - A!, "Bx," Cx (p=1/6)
      3.2 - A!, Bx, "Cx"(p=1/6)

      The x indicates the prisoner dies.  The ! indicates the prisoner is pardoned.  The "" indicates what the warden tells A.

      If the warden says "Bx," there is a 1/3 probability of scenario #1 and a 1/6 probability of scenario #3-1.  (We've ruled out Scenario #2 and #3-2.)  So it seems A's chances of execution are doubled.  But in fact, A's chances of being pardoned have not been halved by his having asked the question.  He had a 1/3 chance of being pardoned regardless (3-1 and 3-2 total 1/3).

      The better way to look at it is that there's an equal probability that the warden will say "Bx" or "Cx," as the probabilities of the warden saying either is 1/2: 1/3 if the other not-A has been pardoned plus 1/6 if A has been pardoned.  Neither response tells A anything about his own fate.

      Good morning! ::hugggggggggggs::

    •  What's changed is just the knowledge (2+ / 0-)
      Recommended by:
      plf515, NCrissieB

      Prisoner A's knowledge of what will happen has changed, he just can't do anything about it.

      Before:

      A, B and C each have:
      1/3 chance of being pardoned.
      2/3 chance of being executed.

      After:

      A has a 1/3 + 2/3 chance, same as before
      B has a 100% chance of being executed
      C has a 2/3 chance of being pardoned and a 1/3 chance of being executed.

      If A could switch bodies with C, he should.  But he can't.

  •  What I constantly find fascinating about the MHP (3+ / 0-)
    Recommended by:
    plf515, NCrissieB, Toon

    Is that whenever it's discussed, including in this thread, people mostly describe it as a problem in statistics rather than information theory or thermodynamics.

    To me (non-mathematician but with a little bit of background in info theory and thermodynamics) the thing that matters is that Monty has told you something by opening the door you didn't choose.  The entropy of the system has changed. It's not so much that there are two distinct statistical distributions posing as one, but that you actually learn something about the correct distribution when Monty opens the door.

    Before any doors are opened you can just pick a door at random, because you don't "know" anything.  Once Monty opens the door he has -- literally -- reduced the entropy in your "view" of the doors by one bit.  The distribution of probabilities itself hasn't changed.  It's your knowledge about the distribution of those probabilities that changed.  Monty has shared with you something that he knew but you didn't (namely that the door he opened was not the correct door).  Now you both know, therefore you can choose your door more wisely.

    The best illustration of how important that is (and the only thing more fascinating than discussions of the MHP ;-) is the difference between the Monty Hall Problem and the Two Children problem.  

    In the Monty Hall problem, the answer is absolutely unambiguous, because it's clear from the narrative that you have acquired a new bit of information.  In the Two Children problem, whether or not you acquire a new bit of information from the mother depends on how the story is told, and interpreted.  Unlike MHP it's ambiguous by default.  In order to tell whether you've actually learned something from the mother you have to introduce extra details to the story and resolve the ambiguity, but those details don't fit into the narrative well.  That leads to fun discussions too and otherwise-sensible people will insist (wrongly!) that there's an unambiguous answer (like in MHP) regardless of how you tel the story.  Even though it depends both on the exact wording and also on whether you're a Bayesian or a frequentist.  :-)

    •  The Two Children problem is language-dependent. (2+ / 0-)
      Recommended by:
      radish, plf515

      A woman has two children.  One of them is a girl.  What is the probability that the other is a boy?  Welllllll ... good luck.  The sex of the girl does not predict the sex of the other child.  Knowing one child is a girl seems to rule out one of four possible sex combinations - that she has two boys - but the sex of the second child was never dependent on the sex of the first.  In fact, knowing one child is a girl rules out four of eight possible sex combinations, and of the four remaining it's 50/50 whether the other child is a boy or girl.

      There are eight equally likely combinations, where the firstborn child is Capitalized and the one you know about is in (parentheses): {(G)g, G(g), (G)b, G(b), (B)g, B(g), (B)b, B(b)}.

      I've bolded the combinations that are still possible given what you know (one child is a girl) and italicized the excluded combinations.  Of those still possible, there are two combinations where the other child is a girl and two where the other child is a boy.  It is 50/50, given the problem as stated.

      Good morning! ::huggggggggggggs::

      •  I think you've over-complicated it ... (3+ / 0-)
        Recommended by:
        radish, plf515, NCrissieB

        There are four possibilities:

        She had two sons
        She had a son, then a daughter
        She had a daughter, then a son
        She had two daughters

        (pretend these are all equally likely outcomes.  they aren't, but that's biology for you.)

        Knowing that the woman has at least one daughter lets you eliminate the 'two sons' option, leaving you with two of three outcomes being 'a son' and one of three outcomes being 'another daughter'.

        Of course, only people from Planet Zog tell you 'I have two children and one is a girl' if the other one is also a girl.

        Probability is hard.  If you haven't already, try the problem in http://blag.xkcd.com/...
        I couldn't figure out why the answer wasn't 5/36.  The explanation makes sense, once I gave up and read the comments for the solution.

        •  The 'gotcha' answer equivocates birth order. (2+ / 0-)
          Recommended by:
          radish, plf515

          The 'gotcha' answer equivocates on birth order.  It says birth order is irrelevant if the other sibling is a girl (girl-girl), but birth order is relevant if the other sibling is a boy (boy-girl, girl-boy).  To be consistent, you must treat birth order as either relevant or irrelevant, regardless of the sibling's sex.  If you treat it as relevant, you have three binary variables (older/younger, boy/girl, sex is known/unknown), and that yields a sample space with eight equal possibilities.  To simplify the notation, the children are listed in birth order, and a Capital letter indicates the sex is known:

          {Gg, gG, Gb, gB, Bg, bG, Bb, bB}

          Here the problem states "one child is a girl," so we can ignore the four combinations with a Capital-B, and we get:

          {Gg, gG, Gb, bG}

          In plain language, that translates to:

          1. The older child (sex is known) is a girl, and the younger child (sex is unknown) is also a girl.
          1. The younger child (sex is known) is a girl, and the older child (sex is unknown) is also a girl.
          1. The older child (sex is known) is a girl, and the younger child (sex is unknown) is a boy.
          1. The younger child (sex is known) is a girl, and the older child (sex is unknown) is a boy.

          Of the four combinations, there are two where the sex of the unknown (older or younger) child is a girl, and two where the sex of the unknown (older or younger) child is a boy.  To conflate #1 and #2 while distinguishing #3 and #4, as the 'gotcha' answer does, equivocates on the relevance of birth order.

          •  I don't see what's wrong with your argument (3+ / 0-)
            Recommended by:
            radish, plf515, NCrissieB

            But I can present a more rigorous approach to my argument:

            Assuming Bayes' Theorem to be true,

            p(H | D) = p(D | H) x p(H) / p(D)

            Here, our Hypothesis is that we have two girls.
            Our Data is that we have at least one girl.
            The probability of our Data, given our Hypothesis (the probability of having at least one girl, given that we have two girls) is 1.
            The probability of our Hypothesis being true is 1/4 (GG, gb, bg, bb).
            The probability of our Data being true is 3/4 (GG, GB, BG, bb)

            1 x (1/4) / (3/4) is 1/3.

            I'm sorry that I can't point out what's wrong with your proof exactly.  I think that would be more convincing than jumping to heavy-duty maths.  My probablistic intuition is lousy, which is why I don't play cards professionally.

            I wonder if your argument founders on "we can ignore the four combinations with a Capital-B".  Those options are eliminated, but what does eliminating them do to the other probabilities?

            •  Ah, but Bayes theorem is only true (2+ / 0-)
              Recommended by:
              plf515, NCrissieB

              for some definitions of "true." :D

              That's the whole point.  You can't point out what's wrong with the proof because there isn't anything "wrong" with it.  It just uses a different assumption about the starting entropy of the system, before anything has been measured.

              In the Bayesian view there are four mutually exclusive uniformly distributed outcomes, so information about one child is information about the entire system. In the frequentist view there are two distinct sets of mutually exclusive outcomes (two in each set), so acquiring information about one XOR does not, by itself, tell you anything about the outcome of the other XOR.

              In order to get the 2/3 answer you have to represent the two-child system as a one-dimensional space containing four mutually exclusive and uniformly distributed possibilities. In order to get to the 1/2 answer you have to represent it as a two-dimensional space containing two discrete sets of two mutually exclusive possibilities.

              How many dimensions you assume is what determines how much significance you can attach to a new "bit" of information that you acquire.

              At the end of the day though, the reason this is so fun is that both ways are perfectly defensible. It's like arguing about who makes the best pizza or whether Milwaukee makes better power tools than DeWalt.

            •  That's a problem statement: (2+ / 0-)
              Recommended by:
              radish, plf515

              If we know "one child is a girl," then by the definition of the problem we must exclude the combinations where "known sex" is a boy.  That's half of the sample space, and leaves the four combinations I listed.

              Your approach, while mathematically correct, has a linguistic equivocation on birth-order.  It treats older-known-younger-unknown and older-unknown-younger-known as identical where both are girls, on the assumption that if both children are girls it doesn't matter whether you know the sex of the older or younger child.  And that's reasonable, except that birth order clearly does matter where the unknown sibling is a boy (older-known-girl-younger-unknown-boy vs. older-unknown-boy-younger-known-girl) because of the probability sample space (two ways to have one-of-each vs. one way each to have two girls or two boys).

              For logical consistency, birth order must matter in both cases or in neither case.  In terms of probability, birth order must matter for a sibling brother.  So the best solution is to apply birth order throughout.  My sample space includes the correct probability for two sisters, sister/brother, two brothers, and birth order is relevant in all cases.

              When you do that, you get the "intuitive" - and biologically correct! - result that you can't predict the sex of one child by the sex of his/her sibling.

              •  Another way to consider it: (2+ / 0-)
                Recommended by:
                radish, plf515

                Change the problem definition a bit.  "Mary has a daughter.  Mary has just learned she's pregnant.  What is the probability that Mary's next child will be another girl?"

                With some narrow biological exceptions, the fact that "Mary has a daughter" doesn't change the probability for the sex of Mary's second child.  So why should it change the probability for an already-born child?

                It doesn't, if you consider birth order relevant in all cases.  It's only when you ignore birth order if both siblings are girls (or boys) that you reach the 'gotcha' answer.

  •  plf, I haven't read all the comments, I have (1+ / 0-)
    Recommended by:
    NCrissieB

    a paper due in a few hours that is, well, incomplete, so I need to go focus on that right now; but I wanted to check back in and thank you for a clear and cogent presentation of a fantastic and fun problem to dig into.

    Math is not my thing. I wish it was more than it is.

    I appreciate the clarity you bring to discussions of mathematical and statistical thinking. I always leave your diaries having learned something. And for me, that is everything.

    I found the discussions, or what I gleaned from skimming them, of Marilyn Vos Savant to be as engaging as the problem itself. If I had more time, as a trained specialist in the education of the gifted and a Mensa member, I think that's where I would have engaged the conversation.  But, I have to go be responsible.

    Thanks for the shout out on the video clips. I didn't know what you'd make of them, but given the way some commenters in these threads (ok, one particular commenter!) chose to engage the topic and its discussants I'm glad we were able to present a different model for constructing meaning.

    Looking forward to tomorrow. Tough act to follow, though. Tough act to follow...

    :-)

    Rogues are preferable to imbeciles because they sometimes take a rest. - Alexandre Dumas

    by elropsych on Tue May 05, 2009 at 09:49:09 AM PDT

  •  Here's how.. (2+ / 0-)
    Recommended by:
    plf515, NCrissieB

    ...it finally made sense to me.

    Pretend it's a magic trick.  Or a street con.  You pick a door, and the sharp makes his offer to you to switch.  He can do this one of three ways:

    1. Offer you a switch to any door.  Why switch, it's the same odds with each one.
    1. Offer you the two other doors in exchange for yours.  You double your odds, so you switch every time.
    1. Do the reveal, which makes it look like he's offering #1 when he's really offering #2.

    Opening the door is a classic diversion.  It looks like he's giving you information, but he's really hiding what the offer is.

  •  Damn, I missed this one (2+ / 0-)
    Recommended by:
    plf515, NCrissieB

    but lots of good comments and discussion on a much misunderstood question. I like the 10 or 100 doors variant myself for explaining it.

    This is not a sig-line.

    by Joffan on Tue May 05, 2009 at 10:13:08 PM PDT

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site