the mysterious inefficacy of weather

I like to think of myself as a data-respecting guy–by which I mean that I try to follow the data wherever it leads, and work hard to suppress my intuitions in cases where those intuitions are convincingly refuted by the empirical evidence. Over the years, I’ve managed to argue myself into believing many things that I would have once found ludicrous–for instance, that parents have very little influence on their children’s personalities, or that in many fields, the judgments of acclaimed experts with decades of training are only marginally better than those of people selected at random, and often considerably worse than simple actuarial models. I believe these things not because I want to or like to, but because I think a dispassionate reading of the available evidence suggests that that’s just how the world works, whether I like it or not.

Still, for all of my efforts, there are times when I find myself unable to set aside my intuitions in the face of what would otherwise be pretty compelling evidence. A case in point is the putative relationship between weather and mood. I think most people–including me–take it as a self-evident fact that weather exerts a strong effect on mood. Climate is one of the first things people bring up when discussing places they’ve lived or visited. When I visit other cities and talk to people about what Austin, Texas (my current home) is like, my description usually amounts to something like it’s an amazing place to live so long as you don’t mind the heat. When people talk about Seattle, they bitch about the rain and the clouds; when people rave about living in California, they’re often thinking in no small part about the constant sunshine that pervades most of the state. When someone comments on the absurdly high rate of death metal bands in Finland, our first reaction is to chuckle and think well, what the hell else is there to do that far up north in the winter?–a reaction promptly followed by a twinge of guilt, because Seasonal Affective Disorder is no laughing matter.

And yet… and yet, the empirical evidence linking variations in the weather to variations in human mood is surprisingly scant. There are a few published reports of very large effects of weather on mood going back several decades, but these are invariably from very small samples–and we know that big correlations tend to occur in little studies. By contrast, large-scale studies with hundreds or thousands of subjects have found very little evidence of a relationship between mood and weather–and the effects identified are not necessarily consistent across studies.

For example, Denissen and colleagues (2008) fit a series of multilevel models of the relationship between objective weather parameters and self-reported mood in 1,233 German subjects, and found only very small associations between weather variables and negative (but not positive) affect. [Klimstra et al (2011)] found similarly negligible main effects in another sample of ~500 subjects. The state of the empirical literature on weather and mood was nicely summed up by Denissen et al in their Discussion:

As indicated by the relatively small regression weights, weather fluctuations accounted for very little variance in people’s day-to-day mood. This result may be unexpected given the existence of commonly held conceptions that weather exerts a strong influence on mood (Watson, 2000), though it replicates findings by Watson (2000) and Keller et al. (2005), who also failed to report main effects. –Dennisen et al (2008)

With the advent of social media and that whole Big Data thing, we can now conduct analyses on a scale that makes the Denissen or Klimstra studies look almost like case studies. In particular, the availability of hundreds of millions of tweets and facebook posts, coupled with comprehensive weather records from every part of the planet, means that we can now investigate the effects of almost every kind of weather pattern (cloud cover, temperature, humidity, barometric pressure, etc.) on many different indices of mood. And yet, here again, the evidence is not very kind to our intuitive notion of a strong association between weather and mood.

For example, in a study of 10 million facebook users in 100 US cities, Coviello et al (2014) found that the incidence of positive posts decreased by approximately 1%, and that of negative posts increased by 1%, on days when rain fell compared to days without rain. While that finding is certainly informative (and served as a starting point for other much more impressive analyses of network contagion), it’s not a terribly impressive demonstration of weather’s supposedly robust impact on mood. I mean, a 1% increase in rain-induced negative affect is probably not what’s really keeping anyone from moving to Seattle. Yet if anyone’s managed to detect a much bigger effect of weather on mood in a large-sample study, I’m not aware of it.

I’ve also had the pleasure of experiencing the mysterious absence of weather effects firsthand: as a graduate student, I once spent nearly two weeks trying to find effects of weather on mood in a large dataset (thousands of users from over twenty cities worldwide) culled from LiveJournal, taking advantage of users’ ability to indicate their mood in a status field via an emoticon (a feat of modern technology that’s now become nearly universal thanks to the introduction of those 4-byte UTF-8 emoji monstrosities 🙀👻🍧😻). I stratified my data eleventy different ways; I tried kneading it into infinity-hundred pleasant geometric shapes; I sang to it in the shower and brought it ice cream in bed. But nothing worked. And I’m pretty sure it wasn’t that my analysis pipeline was fundamentally broken, because I did manage (as a sanity check) to successfully establish that LiveJournal users are more likely to report feeling “cold” when the temperature outside is lower (❄️😢). So it’s not like physical conditions have no effect on people’s internal states. It’s just that the obvious weather variables (temperature, rain, humidity, etc.) don’t seem to shift our mood very much, despite our persistent convictions.

Needless to say, that project is currently languishing quite comfortably in the seventh level of file drawer hell (i.e., that bottom drawer that I locked then somehow lost the key to).

Anyway, the question I’ve been mulling over on and off for several years now–though, two-week data-mining binge aside, never for long enough to actually arrive at a satisfactory answer–is why empirical studies have been largely unable to detect an effect of weather on mood. Here are some of the potential answers I’ve come up with:

  • There really isn’t a strong effect of weather on mood, and the intuition that there is one stems from a perverse kind of cultural belief or confirmation bias that leads us all to behave in very strange, and often life-changing, ways–for example, to insist on moving to Miami instead of Seattle (which, climate aside, would be a crazy move, right?). This certainly allows for the possibility that there are weak effects on mood–which plenty of data already support–but then, that’s not so exciting, and doesn’t explain why so many people are so eager to move to Hawaii or California for the great weather.

  • Weather does exert a big effect on mood, but it does so in a highly idiosyncratic way that largely averages out across individuals. On this view, while most people’s mood might be sensitive to weather to some degree, the precise manifestation differs across individuals, so that some people would rather shoot themselves in the face than spend a week in an Edmonton winter, while others will swear up and down that it really is possible (no, literally!) to melt in the heat of a Texas summer. From a modeling standpoint, if the effects of weather on mood are reliable but extremely idiosyncratic, identifying consistent patterns could be a very difficult proposition, as it would potentially require us to model some pretty complex higher-order interactions. And the difficulty is further compounded by strong geographic selection biases: since people tend to move to places where they like the climate, the variance in mood attributable to weather changes is probably much smaller than it would be under random dispersal.

  • People’s mood is heavily influenced by the weather when they first spend time somewhere new, but then they get used to it. We habituate to almost everything else, so why not weather? Maybe people who live in California don’t really benefit from living in constant sunshine. Maybe they only enjoyed the sun for their first two weeks in California, and the problem is that now, whenever they travel somewhere else, the rain/snow/heat of other places makes them feel worse than their baseline (habituated) state. In other words, maybe Californians have been snorting sunshine for so long that they now need a hit of clarified sunbeams three times a day just to feel normal.

  • The relationship between objective weather variables and subjective emotional states is highly non-linear. Maybe we can’t consistently detect a relationship between high temperatures and anger because the perception of temperature is highly dependent on a range of other variables (e.g., 30 degrees celsius can feel quite pleasant on a cloudy day in a dry climate, but intolerable if it’s humid and the sun is out). This would make the modeling challenge more difficult, but certainly not insurmountable.

  • Our measures of mood are not very reliable, and since reliability limits validity, it’s no surprise if we can’t detect consistent effects of weather on mood. Personally I’m actually very skeptical about this one, since there’s plenty of evidence that self-reports of emotion are more than adequate in any number of other situations (e.g., it’s not at all hard to detect strong trait effects of personality on reported mood states). But it’s still not entirely crazy to suggest that maybe what we’re looking at is at least partly a measurement problem—especially once we start talking about algorithmically extracting sentiment from Twitter or Facebook posts, which is a notoriously difficult problem.

  • The effects of weather on mood are strong, but very transient, and we’re simply not very good at computing mental integrals over all of our moment-by-moment experiences. That is, we tend to overestimate the  impact of weather on our mood because we find it easy to remember instances when the weather affected our mood, and not so easy to track all of the other background factors that might influence our mood more deeply but less perceptibly. There are many heuristics and biases you could attribute this to (e.g., the peak-end rule, the availability heuristic, etc.), but the basic point is that, on this view, the belief that the weather robustly influences our mood is a kind of mnemonic illusion attributable to well-known bugs in (or, more charitably, features of) our cognitive architecture.

Anyway, as far as I can tell, none of the above explanations fully account for the available data. And, to be fair, there’s no reason to think any of them should: if I had to guess, I would put money on the true explanation being a convoluted mosaic of some or all of the above factors (plus others I haven’t considered, no doubt). But the proximal problem is that there just doesn’t seem to be much data to speak to the question one way or the other. And this annoys me more than I would like. I won’t go so far as to say I spend a lot of time thinking about the problem, because I don’t. But I think about it often enough that writing a 2,000-word blog post in the hopes that other folks will provide some compelling input seems like a very reasonable time investment.

And so, having read this far—which must mean you’re at least vaguely entertained, right?—it’s your turn to help me out. Please tell me: Why is it so damn hard to detect the effects of weather on mood? Make it rain comments! It will probably cheer me up. Slightly.

☀️🌞😎😅

19 thoughts on “the mysterious inefficacy of weather”

  1. It’s a measurement problem. Your instinct is right on that weather influences mood because you do tend to feel a little positive boost on sunny days and angst of crapdome on rainy ones, but in both cases will still post an annoyingly positive blubber of emoticon nonsense on social media. I’m not sure how to pick up sadness (or even poignant neutrality) on social media – perhaps absence off a post could be an indicator, but business or (gasp) more involvement with real life could indicate the same thing. It would probably be more telling to post a hidden camera in a public place and compare the frequency of smiling on nice versus crappy days. I still contend that I’m much happier when I wake up and it’s warm and nice outside, although I don’t have any hard evidence to prove it.

    1. I think measurement issues could explain underestimated effects in studies that rely on automated sentiment analysis, but that can’t explain the lack of effect in studies that rely on self-reported mood (which is still most of them). If you posit that you’re much happier when you wake up on happy days, there’s no reason why you wouldn’t be able to reliably log that in a diary…

      1. There is technically no reason why you could not, but there is a very strong social pressure that would lead you to under report any kind of negative sentiment. If we did a survey of the US population for self-reports of happiness, I would bet that most would rate themselves to be happy. But we know that depression is pretty common. Where are all these depressed people? They hide it because they are managing how others perceive them, and to be perceived as unhappy is to go against how we are supposed to be. So, any sort of such a report is more a reflection of an individual wanting to demonstrate (to themselves, and the researcher) being a happy person. In my mind, the only potentially useful way to measure happiness is through variables that might reflect it that are just under awareness, and that is why I mention “spying for smiles.” So I don’t think that the issue is not being able to report happier on sunny days, it’s being able to report neutrality or melancholy on cloudy ones.

        1. No, people don’t seem to have much trouble reporting negative affect in surveys. I mean, there’s undoubtedly some amount of positivity bias due to social desirability etc., but that doesn’t matter in most cases because it’s the relative differences that we care about. In this case, the analysis would work just fine even if most people shift from “I’m not blue at all” to “I’m a teensy little bit blue”. But if you look at the mountains experience sampling data we have, most people have no compunction about reporting being in negative emotional states much of the time (and the argument has also been made that for a sizeable minority of people–e.g., those who are highly neurotic–there may actually be a tendency to overreport negative affect).

          1. That argument is fair. I guess “it would work just fine,” but it still goes against my intuition. Oh well. This is why I like working with concrete, tangible things! :O)

    2. I think this is an intelligent response to an intelligent article. I think it’s far more likely that you can infer whether I am hot or cold from social media than whether I am in a bad mood. I suspect too that in a real sense mood (at least as affected by the weather) is more fleeting and more likely to be pushed around by a suite of factors.

      What might be interesting is to look at people’s mood and their *memories* of weather. When I think of being happy in the past — e.g., as an undergraduate in Scotland — the sun was always shining (though in fact it wasn’t).

  2. Interesting post. I agree that the explanation is probably a mixture of all those factors you mention, but I’d also say that habituation probably plays a very dominant role. I don’t find that very inconceivable. Human beings are very adaptable hence you find human populations in such diverse places like the Amazon, the Sahara, and Siberia.

    The weather can presumably exert a very transient effect of mood but it quickly levels off. It’s difficult to test this. I have plenty of anecdotal evidence from my own life where quick trips to sunny places improved my mood dramatically but they are also entirely confounded by the fact that these quick trips were obviously also exciting for a whole lot of other reasons (not least of all the fact that I could escape the salt mines for a while). So this can’t really be used.

    There are almost certainly also non-linearities at play. If anything a warm temperate climate should be better than extremes at either end. If you ask people from places on the Arabic Peninsula they will tell you that they actually love Welsh weather with all that rain. Of course there may be other factors there too (lack of alcohol?) and this must no doubt also be idiosyncratic.

    Anyway, it’s an interesting question!

  3. Great post! Two thoughts as I read this. First, what about Schwarz & Clore 1983? They take it as a given that people will report worse life satisfaction on rainy days, found exactly that, and then used an attribution manipulation to undo the weather effect. So, do you think that study would not replicate? Second, maybe it’s the case that people’s moods habituate to weather, only being influenced as the weather changes from nice to crap, or crap to nice. Or the opposite, they are only influenced by weather after being bogged down by mother nature’s crap for 5 days. Or, going with S & C, maybe the trick to seeing this correlation is to ask mood questions in such a way that people are encouraged to think about how much they are liking (hating) the weather at that moment. In which case they are not really reporting their mood…

    1. Thanks! The Schwarz & Clore study was very small. There have been quite a few much larger studies that found essentially no effect–e.g., Lucas & Lawless (2013), which found no meaningful effect of weather on life satisfaction ratings in a sample of over a million people (and they review other studies with similar null results). So in all likelihood, yes, the Schwarz & Clore effect was overstated, and the effect of weather on life satisfaction is actually probably quite close to zero–at least under naive conditions (it may still be that if you direct people to think about the weather, they erroneously integrate that into their life satisfaction judgment, but that would presumably tell us something about biases in judgment, rather than about the effects of weather on mood).

  4. Nice post, there is work in the social sciences that observe macro level correlations between weather and behavior – particularly violent behavior and heat. My first morbid thought was that suicides should increase in the rain (people in New York give Ithaca as an anecdotal example) – and I see in a quick google scholar search someone has done that.

    Effect sizes are quite tricky things to wrap your head around. It requires a referent class that can change in the eye of the beholder. I would think in a counter-factual world rain on May 19th versus no rain on May 19th a 1% change is small, but reasonable. There are other potential referents though, e.g. snow in December versus sunshine in July. The former is much easier to assess in a study design.

    This also has implications for the different studies. The effect size is partly a function of the variation in the explanatory variables. For a study within one particular area, there simply may not be enough variation in weather between individuals to detect an effect. Was your study repeated measures of the same individuals, or a cross-sectional design?

    1. Thanks for the reminder–I had actually meant to mention the macroscale effects in my post, but forgot. I agree that these provide some nice evidence of at least weak relationships between weather and mood, though I think the implications are not entirely clear (both because of confounding issues, and because the transfer function relating individual mood changes to increased incidence of crime is not so clear).

      I also agree that a change of 1% for rain vs. no-rain is quite meaningful in the aggregate–but I think it still doesn’t square so well with most people’s intuitive sense that the effect of weather on their mood is much bigger than that.

      With respect to the analyses I did, I’m not entirely sure (they were done about six years ago now, and I have no inclination to dig them up), but I think they were repeated-measures (i.e., status updates nested within individuals nested within cities).

  5. Maybe your bulletpoint no. 4: “relationship between objective weather variables and subjective emotional states is highly non-linear”, might be near an explanation. I don’t know about the investogations you mention, but high and low pressure might be a lot more determining than sun vs. rain. Rain can be very pleasant, and clouds can be very beautiful. The wind can be quite annoying. In a storm you don’t think of mood, you just concentrate on surviving. A farmer is happy when it rains, but not a the time, when he is going to harvest his crops. If you seek advice in music you can find both “Summertime blues” as well as “only happy when it rains.”

  6. I think this point “People’s mood is heavily influenced by the weather when they first spend time somewhere new, but then they get used to it.” is very salient.

    As I was reading your post the words “self-selection” kept ringing in my head – at least in the US, people relocate to places where they find the climate at least above some misery baseline. For instance, I live in Chicago but I don’t think I could ever live in Minneapolis – slightly longer harsher winters. This is about as far north as I can go.

    I would hypothesize that if we took people and randomly placed them around the US – we may be able to observe weather on mood in data. I wonder if you could get a sample of medical residents or some field where people just end up placed somewhere that is not their preference?

  7. The most recent studies of behavioral effects stemming from the occurrence of the full moon may prove informative to your approach. those studies point out the average loss of sleep as the tie between the full moon and the odd behaviors associated only by anecdote before the causal factor became understood.

    Intellicast produces a forecast of attentiveness, focus and discomfort attributable to weather. Perhaps the moods of the populations subject to those forecast, or otherwise measured conditions could be better isolated and tied to the causes to then become more precisely associated with the more specific effects.

  8. Interesting post! Has made me think about how to best model this relationship – it’s a bit of a challenge.

    Have studies taken into consideration the temperatures that individuals experience indoors? I would expect to find a smaller effect of the weather on mood in those who tend to spend their days indoors with the thermostat on the same setting each day.

    Also: 30C is never pleasant! Give me 0C over that any day.

  9. Just speaking from my own experience, I am only aware of feeling good or bad regarding weather when I am aware of it as weather. If I am inside or have been in said weather state for more than a few minutes, my mood appears to be ruled by other things that I am attending to. Frankly I find it surprising to learn that smart people thought there should be large effects of weather on mood. This is as distinct from, say, length of day, which I do think has a background effect on many or most people, including me.

    1. This is a good point too. Daylight hours (and perhaps intensity too – although that again should correlate at least somewhat with weather) might seem more likely to affect mood.

      I also wonder if any relationship can only be found in places with temperate climate. In extremely cold or hot places people tend to try to spend most of their time inside. A friend of mine from Portugal once told me she appreciated that about the UK. As soon as the sun comes out people do their best to enjoy it. This doesn’t typically happen in many “hot” countries where there often aren’t even restaurants/bars where one can sit outside.

  10. Well written, Tal.
    Three comments:
    1. Daniel Kahneman had s study titled “Does living in California make people happy”, which is exactly about the availability heuristic’s role in overestimating the impact of weather (and other things like winning the lottery) on well being: http://pss.sagepub.com/content/9/5/340.short .

    2. Indeed there is a selection bias (people don’t tend to live in places where they hate the weather). Therefore, I love the idea of Tom to investigate medical residence (though the selection bias might occur there as well).

    3. One issue which is completely overlooked by the literature (and is worth checking rigorously)) is that weather consists of many measurable variables — each of them (and their interaction) may have a small influence. It is plausible that all of these variables JOINTLY have a strong influence on mood. If only a dummy variable of rain/no rain has an impact of 1% on an active behavioral measure, that wouldn’t surprise me at all. Obviously the standard statistical tools are inappropriate for studying this rigorously, but tools from machine learning may allow to determine whether many small influences of weather related variables might have predictive power.

Leave a Reply