Introduction
Health claims in the media: part of the problem
Healthcare students and professionals, as well as patients and everyone else, are exposed to countless health claims—particularly claims about the effects of interventions—spreading further and faster than ever, via the Internet. Many of the claims are unreliable, such as those that conflate correlation and causation.1 2 Meanwhile, many people are unable to critically assess their reliability.
For example, here in Norway, a survey conducted in 2019 among a representative sample of the population—including healthcare professionals—indicated that a majority of Norwegians are unable to apply several fundamental concepts for assessing health claims and making informed health choices, such as the importance of similar comparison groups for finding intervention effects (149 of 771 participants were able).3
The combination of unreliable claims and inability to critically assess those claims can lead to uninformed choices (including shared decisions) and be a barrier to evidence-based healthcare (EBHC). Logically, this is a major explanatory factor in the known, worldwide overuse of ineffective and harmful medical services4 and underuse of effective services.4 5
Part of the solution?
However, can the abundance of health claims in the media also be a resource for teaching EBHC? News stories, social media posts and advertisements are simple, familiar, relatable and entertaining, by design, as opposed to scientific literature, which typically includes jargon and excludes narrative. Therefore, health claims in the media may be an appropriate place to start for inexperienced students in the health sciences, as well as other non-professionals, when learning how to think critically about health information.
In this article, we describe the development and large-scale implementation of an educational intervention that systematically takes advantage of health claims in the media to help university students learn how to apply Informed Health Choices (IHC) Key Concepts.6 These concepts are as essential for making informed personal health choices, as for providing evidence-based care. This is reflected in the IHC framework being used in the development of core competencies in evidence-based practice for health professionals.7
We believe that learning to critically assess health claims in the mass media can be a steppingstone for students towards being able to critically appraise scientific literature, including research articles and clinical guidelines. Moreover, the use of the IHC Key Concepts framework can facilitate the adaption of our intervention to various contexts and fields, given so many concepts also apply to other fields.8
Limitations
We have not yet conducted any formal study, due to limited resources, and make no claim about the effects of our intervention. However, we believe the background for the intervention; our work so far; and our plans for implementation and further development and evaluation will be of interest to the audience of this special issue on education in evidence-based healthcare.
Moreover, the intervention was developed by an interdisciplinary group, including students and staff, and it has been piloted several times. In our experience, this is uncommon for educational interventions in EBHC or otherwise. Also, a systematic review has shown there is a lack of interventions that cover particular IHC Key Concepts, cover a large number of IHC Key Concepts and target particular populations.9
Development of the intervention
The project to develop the intervention is named ‘Bak overskriftene’ (https://bakoverskriftene.oslomet.no/about), which is Norwegian for ‘behind the headlines’. A working group with students, teachers and researchers from various study programmes and faculties at Oslo Metropolitan University (OsloMet) have led the development.
The project is inspired by a website with the same name run by the British National Health Service (NHS) (www.nhs.uk/news). It differs in that the primary goal is to help university students master IHC Key Concepts by analysing media information and producing content themselves, as opposed to the NHS site, which provides expert analysis to the general public.
We have iteratively modified and adjusted our intervention, informed by experiences and feedback, and adapted it to different contexts. Through every iteration, the intervention has had two main phases.
First, students introduce themselves to IHC Key Concepts, as per a ‘flipped-classroom’ strategy.10 In this phase, we have taken advantage of two resources: www.thatsaclaim.org, a website dedicated to the dissemination of the IHC framework, and the Teachers of Evidence-Based Health Care Learning Resources Database (www.teachingebhc.org), in which resources can be filtered by IHC Key Concepts, among other variables.
In the second phase, students use health claims in the media to illustrate the concepts themselves, producing short posts (table 1). These are similar to posts on the NHS site in that they have a consistent structure, discuss health information in the media and use plain language, and in that (some) are posted on a dedicated website, that is, the project website (https://bakoverskriftene.oslomet.no). However, they differ in that they focus on explaining a generic concept, as opposed to the specific study or health intervention in question. We give an example in the next section.
Table 1.
Phase 2 of intervention (quoted text translated from Norwegian by the authors)
| Step 1 | Step 2 | Step 3 | Step 4 |
| Student finds or is assigned mass media content | Student extracts claim | Student identifies relevant IHC Key Concept | Student illustrates concept, using the claim |
| Example | |||
‘Norwegian Championship in coffee: 9 good reasons to drink coffee’17
|
‘Coffee can reduce the risk of diabetes Type 2’
|
Correlation ≠ Causation
|
See box 1
|
IHC, Informed Health Choices.
Pilots
We first piloted the intervention as an extracurricular programme at OsloMet over the Spring 2019 semester, with six students from six different study programmes within the Faculties of Health Sciences; Technology, Art and Design; and Social Sciences. The students were selected from a pool of respondents to an open call. Also, in 2019 and 2020, we piloted the intervention within two courses (approximately 20–30 students in each): an elective research communication and science journalism course at the University of Oslo, and a continuing education course in EBHC at OsloMet.
A first series of students’ posts are available on the project website, in Norwegian. Most of the posts originate from the semester-long pilot at OsloMet. In one of the posts, the claim that coffee can reduce the risk of Type 2 diabetes is used to illustrate the difference between correlation and causation.11 Box 1 is a translation of the post. The concept is important for everyone to understand, since associations are the basis for many health claims,2 and particularly important for health professionals when considering the quality of evidence about intervention effects.12
Box 1. Example of a student post (translated from Norwegian by the authors).
In the article from Nettavisen, there are many claims about good reasons to drink coffee. One of the claims is that one more cup of coffee per day can reduce the risk of Type 2 diabetes. The basis for this is that researchers have found a correlation between coffee drinking and fewer cases of Type 2 diabetes.
The article does not say anything about the research, for example, the number of participants, nor anything about where the research was published. Nothing is said about the number of coffee cups you are supposed to drink in total, nor what Type 2 diabetes is.
Regardless, that researchers have observed a correlation between coffee drinking and fewer cases of Type 2 diabetes does not necessarily mean that coffee actually prevents Type 2 diabetes.
Another factor that could explain the difference is, for example, what type of coffee you drink. Those who drink more coffee, could it be that they drink less sugary soft drinks? Maybe those who drink less coffee drink coffee that contains lots of syrup and milk, and has cream on top?
To find out if coffee actually prevents Type 2 diabetes, you need to compare drinking X cups of coffee with something else. Who gets what needs to be chosen at random, so the groups become similar and the groups need to be large enough. In addition, you need to sum up results from all the relevant, reliable comparisons.
The first series of posts on the website went through several rounds of feedback and revisions, controlling the quality of content and style. Two of the authors (MO and MM) provided feedback, while each student revised their own post. MO has been managing editor of the site. MO together with each student made final decisions to publish.
Large-scale implementation
In the Fall 2020 semester, we will adapt and implement the intervention as the first component in a new introductory EBHC course at OsloMet. The course itself will be mandatory for all students in the nine health sciences bachelor programmes (about 1600 per year), in the first 2 years of their degree. Modifications from the original intervention include replacing individual work with work in small groups, as well as allowing for posts in different formats (eg, videos and podcasts), not just text (table 2). We are exploring how to quality-control and publish at least some of the content produced by students in the course.
Table 2.
Implementation of intervention in new introductory EBHC course (under development)
| Course starts | |
| Phase 1 of intervention | |
|
|
|
|
| Phase 2 of intervention | |
|
|
| Course continues | |
| End of intervention |
|
EBHC, evidence-based healthcare; IHC, Informed Health Choices.
Next steps
We are planning a formal, mixed-methods evaluation of the intervention in relation to its implementation in the new course. A randomised trial is not feasible in the first instance due to limited resources. Besides, previous experience suggests qualitative data from focus groups or in-depth interviews are likely to reveal obvious ways in which students’ and teachers’ experience of the intervention can be improved, and a randomised trial would not be sensible until after making such improvements.13 14 Therefore, the initial quantitative evaluation is likely to be a before-after test, using items from the Claim Evaluation Tools database, which have been specifically developed to measure the ability to apply the IHC Key Concepts.15 16
We welcome feedback and collaboration, nationally and internationally.
Acknowledgments
In addition to other current and former members of the 'Bak overskriftene' steering and working groups, and students who have participated in pilots of the intervention, we are grateful to Gro Røkholt and Åsmund H. Eikenes, for facilitating the pilots within courses, and to members of our national and international advisory groups, for their support and advice. Solveig Bua Løken created the illustrations in box 1.
Footnotes
Twitter: @matt_oxman
Contributors: MO drafted the manuscript. LH, GJ, BK, and MM contributed to revisions.
Funding: 'Bak overskriftene' has received funding from the central administration at OsloMet.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
Ethics statements
Patient consent for publication
Not required.
References
- 1.Walsh-Childers K, Braddock J, Rabaza C, et al. One step forward, one step back: changes in news coverage of medical interventions. Health Commun 2018;33:174–87. 10.1080/10410236.2016.1250706 [DOI] [PubMed] [Google Scholar]
- 2.Haber N, Smith ER, Moscoe E, et al. Causal language and strength of inference in academic and media articles shared in social media (claims): a systematic review. PLoS One 2018;13:e0196346. 10.1371/journal.pone.0196346 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Dahlgren A, Furuseth-Olsen K, Rose CJ, et al. The Norwegian public’s ability to assess treatment claims: results of a cross-sectional study of critical health literacy. F1000Res 2020;9:179. 10.12688/f1000research.21902.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Brownlee S, Chalkidou K, Doust J, et al. Evidence for overuse of medical services around the world. Lancet 2017;390:156–68. 10.1016/S0140-6736(16)32585-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Glasziou P, Straus S, Brownlee S, et al. Evidence for underuse of effective medical services around the world. Lancet 2017;390:169–77. 10.1016/S0140-6736(16)30946-1 [DOI] [PubMed] [Google Scholar]
- 6.Oxman AD, Chalmers I, Austvoll-Dahlgren A, et al. Key concepts for assessing claims about treatment effects and making well-informed treatment choices. F1000Res 2018;7:7. 10.12688/f1000research.16771.2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Albarqouni L, Hoffmann T, Straus S, et al. Core competencies in evidence-based practice for health professionals: consensus statement based on a systematic review and Delphi survey. JAMA Netw Open 2018;1:e180281. 10.1001/jamanetworkopen.2018.0281 [DOI] [PubMed] [Google Scholar]
- 8.Aronson JK, Barends E, Boruch R, et al. Key concepts for making informed choices. Nature 2019;572:303–6. 10.1038/d41586-019-02407-9 [DOI] [PubMed] [Google Scholar]
- 9.Austvoll-Dahlgren A, Nsangi A, Semakula D. Interventions and assessment tools addressing key concepts people need to know to appraise claims about treatment effects: a systematic mapping review. Syst Rev 2016;5:215. 10.1186/s13643-016-0389-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Cheng L, Ritzhaupt AD, Antonenko P. Effects of the flipped classroom instructional strategy on students’ learning outcomes: a meta-analysis. Educ Tech Res Dev 2019;67:793–824. 10.1007/s11423-018-9633-7 [DOI] [Google Scholar]
- 11.Larsgård A. Kaffe og diabetes type 2. Bak overskriftene, 2019. Available: https://bakoverskriftene.oslomet.no/kaffe-og-diabetes-type-2 [Accessed 20 Mar 2020].
- 12.Guyatt GH, Oxman AD, Kunz R, et al. What is "quality of evidence" and why is it important to clinicians? BMJ 2008;336:995–8. 10.1136/bmj.39490.551019.BE [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Nsangi A, Semakula D, Rosenbaum SE, et al. Development of the informed health choices resources in four countries to teach primary school children to assess claims about treatment effects: a qualitative study employing a user-centred approach. Pilot Feasibility Stud 2020;6:18. 10.1186/s40814-020-00565-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Semakula D, Nsangi A, Oxman M, et al. Development of mass media resources to improve the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about the effects of treatments: a human-centred design approach. Pilot Feasibility Stud 2019;5:155. 10.1186/s40814-019-0540-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Austvoll-Dahlgren A, Guttersrud Øystein, Nsangi A, et al. Measuring ability to assess claims about treatment effects: a latent trait analysis of items from the 'Claim Evaluation Tools' database using Rasch modelling. BMJ Open 2017;7:e013185. 10.1136/bmjopen-2016-013185 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Austvoll-Dahlgren A, Semakula D, Nsangi A, et al. Measuring ability to assess claims about treatment effects: the development of the 'Claim Evaluation Tools'. BMJ Open 2017;7:e013184. 10.1136/bmjopen-2016-013184 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Stalsberg T. NM i kaffe: 9 gode grunner til å drikke kaffe. Nettavisen, 2017. Available: https://www.nettavisen.no/livsstil/nm-i-kaffe-9-gode-grunner-til-a-drikke-kaffe/3423310463.html [Accessed 20 Mar 2020].