As you may have read, the BBC just published the results of a ‘brain training’ study in Nature magazine. The show “Bang goes the Theory” funded and conducted a study with 8,600 subjects using a simple set of light-weight computer games, played them for a minimal time and declared to the world that “brain games don’t work”. With so many willing subjects involved, it is unfortunate that it was a wasted opportunity to apply depth and rigor to the study and do a serious and responsible job. The ill-founded experiment ultimately throws the baby out with the bath water. It is great propaganda for hyping a popular TV show with sound-bites, however, the hype and rhetoric generated by it is a disservice to those that could benefit from serious scientific work.
To be specific, the study only uncovered that the specific brain games they developed for use in the test were no more efficient than the tasks they used in their control group. The results should not surprise anyone when you consider the tools and methods used. Let’s review a few points here:
1) The duration of the training program was very short (6 tasks for a minimum of 10 minutes a day, three times a week, over a six-week period). In 2008, the BBC contacted HAPPYneuron for early advice in designing the experiment. At the time we advised the BBC that the training duration was too short and that no reportable conclusion could be drawn from an experiment of this duration. This opinion was a direct result of other studies and our own extensive experience and analysis. It is well established that the minimal engagement time to see any statistically significant effect from brain training is 90 days, with 3 sessions per week of at least 30 minutes in duration.
2) The study was seeking to demonstrate ‘transferability’, that is to see a correlation between task improvement and benchmarking improvement. The application of meta-cognition is essential for transferability. It is a complex process (indeed the subject of many a Ph.D. theses) and must be intently and explicitly designed into the training exercises. Two key factors that are necessary to achieve transferability were absent from BBC experiment. These are:
a) a didactic approach which shows the strategies and mechanisms that help develop meta-cognition abilities for improvement
b) a multiplicity of situations (large variety of exercises ) where the acquired meta-cognition abilities can be re-applied in different exercises
The BBC experiment had just 6 exercises, certainly not enough for a subject to reapply the approaches that they may have learned.
3) The control group was an interesting choice and arguably stacked the odds in favor of the conclusion from the onset. To determine if an activity is effective, the ideal situation is to compare the activity with what people are likely to do in real life. Will people systematically and regularly perform the same type of explorative internet searches on an ongoing basis of the caliber that was given to the control subjects? The subjects were asked to explore a set of obscure questions (e.g. in what year did Henry VIII die?) and then to order them chronologically. This required systematic Web research and exercised sequencing skills, language skills, reasoning, working memory and other cognitive functions… arguably a cognitive task at least on par with the training exercises. So we are not surprised to also see improvements within the control group, or a reduction in the difference between the trained groups and the control group. The study’s claim is that “in all three groups the standardized effect sizes of the transfer effects were, at best, small, suggesting that any comparison (even with a control group who did nothing) would have yielded a negligible brain training effect in the experimental groups.” Yes, but all this suggests is that if you jointly use a poor brain training program at a light dose and you are compared against a stimulated control group, you limit the chance to see real improvement! Optimistically, the study might have had the honesty and scientific integrity to conclude “That said, the possibility that an even more extensive training regimen may have eventually produced an effect cannot be excluded.” Lastly, the experiment’s reasoning benchmark is a grammatical reasoning test, in which no training test included language materials (which was not the case of the control group).
To conclude, designing an efficient brain training platform is a complex task that requires many important and essential elements you cannot simply get by compiling basic games and expecting positive results with a minimal training duration. This is the difference between leisure gaming activity and a properly designed brain training program. It is important to note, as shown by many published studies*, brain training software is not the only way of receiving broad cognitive stimulation and preventing cognitive decline. Cognitive stimulation can also be achieved with a highly cognitive stimulated way of life (traveling, gardening, reading, learning, etc). What brain training software offers is another way to achieve this stimulation, a simple and inexpensive solution to compensate for the decrease in cognitive activity that happens to most people as they age.
So, we encourage people interested in the benefit of brain training to see though all of the sound bites and the veneer of what has been this highly publicized conclusion by a poorly designed study and to continue to support the serious cognitive research going on at universities and hospital systems around the world.
Bernard Croisile, MD, Ph.D and the HAPPYneuron scientific team
* Research Studies
5 Comments
Sent to the Bang goes the Theory website:
In your brain games tests, the results showed that there was an improvement for the the group who did the games. The delta was not large, but it was enough to be significant.
The published data, and how its reflected in the media states the games did not have beneficial effects.
However what is not taken into consideration is what results would be over more time, and most significantly what are the degenerative effects over time of those who trained vs, not Brain game players.
Now while these games could just be puzzles without all the hyped effects could still be true…you experiment fails to show this. It draws a conclusion over a short period of time, and in no ways shows what time does to brain as it ages. THESE are the goals of these games.
Your study does not meet the criteria of what users of these games are looking for. Further you publishing of data as a conclusion of the merits of such games, is a disservice to those who would try to use such games.
The design of the study is hopelessly flawed. The conclusion is too broad, and not supported by data. The goals of the study are not the goals of the end-users, and the users are what these games are for.
The use it or lose it argument in human cognition is a sound theory. Designing a study that does not reflect the impacts of time and loss of function, is flawed science.
Regards,
Mr. Wolff
I would be very interested to see references for the statement, “It is well established that the minimal engagement time to see any statistically significant effect from brain training is 90 days, with 3 sessions per week of at least 30 minutes in duration” if available.
I am completely consumed with this question regarding the efficacy of brain training through on-line games. And although I cannot as of yet apply the methods and concepts of Dr. Croisile, I would like to construct a methodology that may be applied that may be applied by the gaming participants.
First, everyone involved in providing the games has a profit motive: only the consumer has what may be considered a transcendent need: to overcome cognitive decline as they age.
And that leads me to consider this: the distinct demographics for game use that we are speaking of (as opposed to violent video games): users will primarily be people of AARP candidacy, or to borrow a term from amateur athletics, ‘masters’ contestants. Thirty-somethings indeed may be invigorated by these games, yet fundamentally if they are intelligent and formally educated I would think this pre-occupation to be somewhat boring.
Now then what we want to consider is ‘transferability’ and ‘plasticity’.
Think: if I take a carpenter and teach him chess do I anticipate that he will use his nail gun any better? There are too many variables at work to effectively isolate him in his job? Furthermore, we must remember that transferability apparently arises from the application of IQ tests: effectively stating that people who score high on these tests frequently perform poorly in everyday life. As my strawman above shows, transferability is a pleasant but unnecessary characteristic to apply to our games.
‘Plasticity’ however is more important. Writers for the past score of years have sought to prove to the public that you can spontaneously ignite neurogenesis particularly in the aging brain. Almost all anecdotal evidence suggests plasticity arises in extremely challenged cortical regions (even this keyboard I write on, junk that it is, does not threaten my very existence.) Real-life existence indeed is threatened.
So, am I to conclude with BGTT that the games are fundamentally useless? No. I am sufficiently apprised that I want to age well and aging well suggests that my brain is indeed a valuable component as to how will interact with my environment, and challenge is a necessary component for brain maintenance. It is time now for BGTT to answer the objections being brought to the forefront by concerned participants in the burgeoning frontiers of aging and the brain. Thanks for the forum.
Hi Shelli, based on the observational studies that we have conducted with the HAPPYneuron program (studies # 5, 9 & 11 listed in our Brain Research Section and others pending) that measurable improvements in the 5 major cognitive categories are seen after training for that amount of time.
I saw part of that show and was very astonished about the “results” as so many other studies show that you need to keep you brain busy and engaged to stay fit and “young”.
I am sure that, like any kind of training, you will get positive results when engaging your brain on a regular basis and challenging it with new tasks. Like the muscles in your body need regular exercise that varies as well.
My conclusion? Don’t believe something is true just because it was shown on TV!