Comments on: in defense of three of my favorite sayings http://www.talyarkoni.org/blog/2010/05/14/in-defense-of-three-of-my-favorite-sayings/ ...or you get no soup for one year! Fri, 10 Feb 2012 12:37:32 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: [citation needed]» Blog Archive » what the Dunning-Kruger effect is and isn’t http://www.talyarkoni.org/blog/2010/05/14/in-defense-of-three-of-my-favorite-sayings/comment-page-1/#comment-1539 [citation needed]» Blog Archive » what the Dunning-Kruger effect is and isn’t Wed, 07 Jul 2010 06:52:20 +0000 http://www.talyarkoni.org/blog/?p=554#comment-1539 [...] endurance, precisely because I can’t finish runs that most other runners find trivial!). But the plural of anecdote is not data, and the data appear to be equivocal. Next time you’re inclined to chalk your obnoxious [...] [...] endurance, precisely because I can’t finish runs that most other runners find trivial!). But the plural of anecdote is not data, and the data appear to be equivocal. Next time you’re inclined to chalk your obnoxious [...]

]]> By: Worldliness http://www.talyarkoni.org/blog/2010/05/14/in-defense-of-three-of-my-favorite-sayings/comment-page-1/#comment-1133 Worldliness Mon, 24 May 2010 16:49:30 +0000 http://www.talyarkoni.org/blog/?p=554#comment-1133 Hi Tal, Since you're a stats guy, I thought I'd share this link with you. Pretty good article on Psychology's most recent methodological problems: (a) the overuse of significance testing, (b) underuse of error correcting, and (c) use of faulty meta-analysis methods. http://pps.sagepub.com/content/5/3/233.full Cheers, Brian www.twitter.com/worldliness Hi Tal,
Since you’re a stats guy, I thought I’d share this link with you. Pretty good article on Psychology’s most recent methodological problems: (a) the overuse of significance testing, (b) underuse of error correcting, and (c) use of faulty meta-analysis methods.
http://pps.sagepub.com/content/5/3/233.full
Cheers,
Brian
http://www.twitter.com/worldliness

]]> By: Tal Yarkoni http://www.talyarkoni.org/blog/2010/05/14/in-defense-of-three-of-my-favorite-sayings/comment-page-1/#comment-989 Tal Yarkoni Fri, 14 May 2010 21:58:13 +0000 http://www.talyarkoni.org/blog/?p=554#comment-989 Well, it <em>is</em> evidence against causation, it's just not <em>conclusive</em> evidence. How strong the evidence against a causal relation will be depends entirely on situational factors. One of them, as you mention, is the possibility of suppression effects. Another is the same issue I alluded to of statistical power: if you run an analysis on 4 people and end up with an effect of exactly zero, you probably shouldn't take that as evidence of anything, really, since your confidence intervals are going to be gigantic. My point wasn't to suggest that when we get an effect size of zero we should automatically conclude there can't be a causal relationship (which is simply accepting the null), it was simply to note that even under conditions when you <em>could</em> draw that inference, it still wouldn't follow that a positive correlation would imply the <em>presence</em> of a causal relationship. But I've changed the wording to say a null effect "could be" strong evidence, and not that it <em>is</em>, which is obviously untrue. Well, it is evidence against causation, it’s just not conclusive evidence. How strong the evidence against a causal relation will be depends entirely on situational factors. One of them, as you mention, is the possibility of suppression effects. Another is the same issue I alluded to of statistical power: if you run an analysis on 4 people and end up with an effect of exactly zero, you probably shouldn’t take that as evidence of anything, really, since your confidence intervals are going to be gigantic. My point wasn’t to suggest that when we get an effect size of zero we should automatically conclude there can’t be a causal relationship (which is simply accepting the null), it was simply to note that even under conditions when you could draw that inference, it still wouldn’t follow that a positive correlation would imply the presence of a causal relationship. But I’ve changed the wording to say a null effect “could be” strong evidence, and not that it is, which is obviously untrue.

]]> By: Neuroskeptic http://www.talyarkoni.org/blog/2010/05/14/in-defense-of-three-of-my-favorite-sayings/comment-page-1/#comment-988 Neuroskeptic Fri, 14 May 2010 21:20:43 +0000 http://www.talyarkoni.org/blog/?p=554#comment-988 Oh and also, imagine a drug that was really effective at stopping you from dying from a horrible disease; on a population level, exposure to that drug would not be correlated with life expectancy, because it works so well that no-one dies of the disease; but it causes you to live much longer. Oh and also, imagine a drug that was really effective at stopping you from dying from a horrible disease; on a population level, exposure to that drug would not be correlated with life expectancy, because it works so well that no-one dies of the disease; but it causes you to live much longer.

]]> By: Neuroskeptic http://www.talyarkoni.org/blog/2010/05/14/in-defense-of-three-of-my-favorite-sayings/comment-page-1/#comment-987 Neuroskeptic Fri, 14 May 2010 21:14:24 +0000 http://www.talyarkoni.org/blog/?p=554#comment-987 Zero correlation is not evidence against causation. It could be that there are multiple causal mechanisms that cancel out. This is considered a real possibility in the antidepressant-suicide debate where there is no consistent correlation between antidepressant use & suicide rates on a population level, but from RCTs (which can prove causation), there's reason to believe that they may increase suicides in young people, and decrease them in older people. (Although personally I don't believe that because the RCTs are underpowered to detect effects on suicide, and have a very low base rate of suicide). Zero correlation is not evidence against causation. It could be that there are multiple causal mechanisms that cancel out. This is considered a real possibility in the antidepressant-suicide debate where there is no consistent correlation between antidepressant use & suicide rates on a population level, but from RCTs (which can prove causation), there’s reason to believe that they may increase suicides in young people, and decrease them in older people.

(Although personally I don’t believe that because the RCTs are underpowered to detect effects on suicide, and have a very low base rate of suicide).

]]>