Grey literature in suicide research and practice
This post is from a presentation delivered by Hayley Purdon at the International Association for Suicide Prevention Research Workshop 2023, hosted by Orygen, Melbourne.
Hayley Purdon presenting at the IASP Research Workshop Australia 2023.
As researchers, we value knowledge that is generated with strict controls and processes in place so that we can be sure about the conclusions that come from our work.
But in practice related fields, like out there in the suicide prevention sector, knowledge can come from many different sources. Organisations rely on their own institutional and practice-related knowledge and also knowledge from the people they support, people with lived experience. More and more we are also seeing organisations use knowledge generated from private consultancy firms with primarily commercial objectives. This knowledge represents grey information and literature and is usually missing from our research.
Various sources of knowledge used by suicide prevention organisations in Australia.
But we need to know what suicide prevention organisations are basing their programs on if we are to truly understand how suicide exists and is prevented in Australia.
Grey literature is often excluded from systematic reviews because of reasons like quality, variability and difficulty of analysis. But it exists and frankly, is probably used more in practice than your research publications are, especially if you publish behind a paywall.
But how do we go about systematically reviewing grey literature? There is very little current specific guidance about searching for and analysing material outside of academic databases and the research that does look at grey literature, very rarely publishes specifics about their searches. We need to start reporting more on grey literature methodologies!
In my approach, I spoke to researchers who had used grey literature and I formed a systematic process of my own using google searches. Without going into the detail, I want you to take away three tips to tame google.
Focus on no more than the first three pages, or top 30 results for each search string.
When you google something, how often do you scroll to the second or third page of the results? I bet it is very rare that you go as far as the fourth page. So conduct your search as people would be doing in practice.
Next, restrict your search items to return .PDF items only.
I looked at the suitability of PDF and non-PDF results for the first 6 search strings in my review. Just under 32% of the the non-PDF results were suitable resources, whereas 70% of the PDF results were suitable.
Lastly, adapt your search settings.
Google has a few features you can play with to overcome the tailoring algorithm and get a search as close to a systematic process as possible.
If we look at my numbers, it presents a compelling case for including grey literature. If I focussed on the peer reviewed papers only, I would have 23 items for inclusion. You might be daunted by the sheer number of items that were returned in the google search but you can see that over 80% of these were duplicates. Despite this, there are still 160 relevant papers that would have been missed if I excluded grey literature. What information could you be missing by excluding grey literature?
Table: Number of resources by type, as they were screened in the research project.
Note: Two grey resources were excluded from the review as they had expired links at the time of full-text screening.
The papers I found are from prominent organisations like Life in Mind, Suicide Prevention Australia, Wesley Mission and Standby Support. There are also items included from each state and national governments in the form of documents from state health policy and things like large scale government reviews.
Both forms of information tell different stories about my research topic. The grey is describing the “how we do things” whereas the peer-reviewed literature is describing the “what we have tested”. Both represent two different parts of the suicide prevention puzzle and they very rarely connect and sometimes have conflicting conclusions.
So in your research practice, I challenge you to think about what the grey literature might say about your focus. By doing this, you start to understand how your research fits into practice.
If you aren’t thinking about how your research fits in to the practice of suicide prevention, I think you need to reassess your research priorities!
So get googling!
