Are experimental studies always best?

Work has begun on our NIHR funded evaluation of Forestry Commission Scotland’s Woodlands In and Around Town (WIAT) scheme. WIAT aims to improve quality of life in towns and cities by bringing neglected woodlands into management, creating new woods and supporting people to use and enjoy their local woods. Our study, led by Catharine Ward Thompson at OpenSpace, is focused on whether changes to the local woodland environment affect people’s health. The WIAT evaluation is exciting partly because it’s a rare opportunity to ask what impact environment has on health, at a population level, via an experimental study.

The vast majority of evidence about how health and behaviour are affected by environment comes from cross-sectional studies. In cross-sectional studies, we measure both the environmental characteristic of interest (for example, how much green space there is in a neighbourhood), and the outcome of interest (for example, how healthy or happy the residents of that neighbourhood are) at the same time. Cross-sectional studies are great for suggesting links or associations between environmental characteristics and health or related behaviour, but they have many problems. In particular, we can’t be certain that the aspect of environment we are interested in causes the health outcome in question. In the case of green space and health for example, we worry that the apparent relationship between access to green space in a neighbourhood and good health among residents is really because the residents of greener neighbourhoods tend to be wealthier, and wealthier people are more likely to be healthier anyway. So, it might be that access to green space in a neighbourhood doesn’t cause better health, it’s just that healthier people are more likely to live in greener neighbourhoods.

Experimental studies are very different. In an experiment, we deliberately alter some aspect of the environment for one group of people (the intervention group), but not for another very similar group of people (the control group). We then compare what happens to health or related behaviour in the intervention and control groups. If health improves in the intervention group, but not in the control group, we can be more certain that the change in environment has caused the change in health. So, in our WIAT study, we’ll be comparing what happens to the health of communities whose woodlands are improved and promoted, with those whose woodlands are not. (That sounds a bit unfair on the ‘control’ communities but, in fact, they’ll be eligible to get their woods improved later).

A lot has been written recently about how important experimental studies are*, how much better they are for telling us ‘what works’ to improve health and behaviour, and how we need far more of them. The idea has taken hold, helped by research funding and by the fact that some key journals in public health and epidemiology now refuse to even peer review studies that are cross-sectional. Jim Dunn and Martin Bobak’s editorial* on taking over the editorship of JECH is a good indication of increased interest in experimental designs from leading journals. Mark Petticrew has also written* about it.

I am excited about the prospect of experimental studies being used to examine the impacts of environment on health and health-related behaviour. I believe that the characteristics of the places we live and work in can be a strong influence on our health and behaviour and, in turn, I think that environment could be an effective lever for improving population health and narrowing health inequalities. Experimental studies are, in theory, the best way of finding out if my ideas are right or not.

However, I do have a few concerns about the assumption that experimental approaches are always best for researching ‘what works’ to improve public health. Their strengths have been highlighted in the literature, but there has been relatively little critical thinking about them.

The processes by which environment influence our health and behaviour are complex and life long. Environment doesn’t simply determine health and behaviour; people and environments influence each other. Think about the cycling infrastructure in Copenhagen for example. The environment there enables and encourages people to cycle, so the city’s high rates of active travel are partly because of the environment. However, the environment is so conducive to active travel because the residents use it, protect it, value it and continue to improve it.

Our relationships with different aspects of environment are also formed over the whole of our lives. Catharine Ward Thompson’s work*, for example, shows that one of the strongest predictors of whether we visit woodlands as adults was whether we did so as children. That means just changing access to woodlands in the neighbourhood may not affect immediately, or at all, residents who don’t have ‘visiting the woods’ as part of their culture.

Do we know how long it will take for an environmental change to affect health and behaviour? My guess is that the time will vary by environmental characteristic and/or the health or behavioural outcome being measured. I think, in many cases, effects will be slow to materialise. Yet the reality of research, and research funding, is that it’s difficult to sustain an experiment for a long time. In turn, this might lead us, or perhaps other less critical audiences, to prioritise interventions on aspects of environment that show a quick effect, at the expense of those which may have a greater but slower effect. Worse, if brief experimental studies find no effect of environmental intervention on health, and we think experimental evidence is the best there is, it may lead to the assumption that environment does not affect health.

I worry that in the rush to use experimental designs to see ‘what works’ for public health, we have forgotten some of what we know about relationships between health and environment specifically, and about relationships between place and identity more broadly.  I think experiments are very important, but I’d like to see a more critical perspective.

What do you think?

*NB links to journal articles may require institutional/personal subscription to the journal

3 thoughts on “Are experimental studies always best?”

  1. I think the study you began talking about, the Forestry Commission Scotland’s Woodlands In and Around Town (WIAT) scheme, should show just how important these outdoor spaces to health in general. One of the problems I have found where I live is a lack of green space, woodland or generally anywhere were you can get away from the concret and smog generated from city life. We need areas such as this, for ourselves as much as for the environment.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s