Why we think things work when they don't... and don't work when they do.

One of the most common reasons for false beliefs comes from the non-intuitive nature of mathematics and statistics.  Often the true nature of randomness is misunderstood, as is the notion of central tendency (that is the propensity of bits of data to cluster around their average).  The main reason for this is that we are hardwired to assign causes to the things that happen around us, even when they happen randomly, or when they are simply a reflection of the natural properties of data.  Why do so many ships and planes get lost in the Bermuda triangle?  Not because it is cursed, but because of randomness (the percentage of vessels lost in the Bermuda triangle is no different when compared to other similarly sized areas).  Why is it that pro football players who grace the cover of the Madden NFL video game appear to suffer from bad luck?  Again, not because they are cursed , but because they have likely had a sustained period of great performance and good luck leading up to this honor, and now, they have nowhere to go but down.  In essence, this latter is an example of a property of data known as regression toward the mean.  For example, many people believe that if a company reports record high earnings for a given period that they should immediately invest in that company because it will continue to improve its earnings.  Or, some very driven people who do exceptionally well on a standardized test like the SAT, LSAT, or MCAT exams may feel a strong desire to re-take the exam to try and bump up their score a little bit. These people are often disappointed when the company they invested in reports less earnings the next quarter, or when the second MCAT or SAT score is actually lower than the first.  The reason for these disappointments may not be due to poor performance, or to "slacking off", but rather to a property of data known as "regression toward the mean".  (And there is a great, and rather intuitive explanation of this property over at neuroskeptic that I highly suggest you read).

Basically, regression toward the mean is a property of data, or a property of things that occur when there is some intuitive level of what can be considered extremely high or extremely low.   Let's take the example of SAT scores.  The scores range from 600 (extremely low) to 2400 (extremely high).  If you were to take the test and get a 2300 (extremely high) you should be content with your score and NOT take the test again.  The reason for this is that data points tend toward the mean, or the average (which is why it is the average).  The trick is, knowing what that average is.  In this case, let's assume you took a test prep course during which you took several practice exams with the average score for those being 1900. If this is true, then taking the actual test again will most likely result in a score that is lower than 2300, and closer to your average score of 1900 (thus regression toward the mean).  Of course, the opposite is just as true, if, for some reason, you get a 1400 on the test, you should definitely re-take it, as you will most likely score higher (and closer to 1900) the next time around.  This is the phenomenon of regression toward the mean, and it is an important thing to be able to recognize because there are many more instances where not recognizing this property can be costly.  When this happens, it is called the regression fallacy, and it can lead us to assign cause to things where there is none.  For example, let's say you suffer from headaches or from back pain.  Most of the time the pain is bearable and so you go about your daily life paying no notice.  BUT, at some point, the pain becomes unbearable... it is extremely high and thus demands your attention.  So you go to the store and you buy some homeopathic "treatment" or you go to the chiropractor, or whatever.  Some time after the "treatment" you feel a little bit better, and thus, you think that the treatment is working, and that it must be the cause of your alleviation.  HOWEVER, what is most likely occurring is simply a regression toward the mean, that is, the pain you are feeling is simply regressing toward your average level of pain.  The cause of your feeling better is simply the nature of things, not necessarily the supposed treatment, and understanding this can help to prevent the unnecessary expense of time and money on treatments that don't really work.
Another classic example of the regression fallacy occurs in parenting.  Most people think that punishments are the only effective way to discipline children.  However, years of research are conclusive in showing that positive reinforcement, that is, rewarding good behavior, is more effective than punishment of bad behavior.  The reason most people don't believe the data is because "good" and "bad" childhood behavior represent a classic example of regression toward the mean and the regression fallacy.  Let's assume that a child's behavior can be quantified and placed on a scale of good and bad, with really good behavior (say bringing you breakfast in bed and the morning paper before quietly going about doing all the daily chores) being a 10 and really bad behavior (say setting the cat on fire) as a zero.  Now, most children will probably maintain some average level of good and bad behavior that hovers around, oh, let's say a 6.  BUT sometimes, the child may do something really bad, say around a 2.  You naturally respond in anger and yell at or even spank the child.  Later, the child is behaving around a 6 again, and so you assume that the punishment worked. HOWEVER, this is most likely NOT the result of the yelling and spanking, but simply the child returning to its average level of behavior (regressing to the mean).  Conversely, let's say you read somewhere that positive reinforcement is the way to go, and want to try it out.  You come home from work, and the cat is thankfully not ablaze, and little Timmy is sitting quietly doing his homework... let's call this an 8 or 9.  You decide to reward Timmy by taking him out for ice cream or letting him stay up a half hour later (of course the critical part for this to work is that you explain to Timmy why you are rewarding him).  Now, you fully expect that every night you will be greeted by Timmy acting on his best behavior.  INSTEAD, however, you come home the very next night and find Timmy running around like a crazy person.  What happened?  That positive reinforcement is a load of crap! Right?  Well, the more likely explanation is that the positive reinforcement is working (though it will likely take time to see real results), and ultimately, Timmy will always regress to his average (and occasionally below average) levels of behavior.  The trick is try and raise that average level over the long term, and to be mindful that periods of extreme behavior (either good or bad) will occur and will almost always be followed by more moderate or average behavior.
So, if you think you've got it, excellent!  If not, or if you'd like to read more, I would suggest the post over at Neuroskeptic to learn more about regression to the mean.  And to learn more about the regression fallacy (and other examples of where it might be at play) I highly recommend this book.  And, if you want to learn more about the power of positive reinforcement and communication in parenting, then I recommend this book.