Morwenna Rogers, Alison Bethel and Amanda Wanner ran a continuing education course at the European Health Information and Libraries (EAHIL) Conference.
The subject of the course?
Supplementary Search Methods: who, why, what, how and when
As a group we decided to publish here what we discussed so thanks go to everyone who took part including: Ralph de Vries, Eva Karin and Chantal Haan
Supplementary searching is carried out as part of a systematic review but the methods are often not reported in a way to be transparent and therefore replicable which are two of the main pillars of systematic reviews.
We started off discussing why you might carry out supplementary searching then looked at different supplementary search methods and what they were, how you might do it, who would do it and when within the project.
Why might we carry out supplementary searching
Not all journals are indexed on the databases you are searching or have access to, some references may not be indexed accurately so your search doesn’t pick them up. Perhaps if you don’t find enough studies from database searching, or, it says it in the protocol. There is evidence in the literature that supplementary search methods do pick up additional studies.
1. Forward and backwards citation chasing/searching (BCC and FCC)
What:
FCC – when you use a list of references to search and find out who else has cited them, often the systematic review included references
BCC – when you look at the end of a reference to find out if they have cited any further relevant references
How:
Can use databases including Web of Science and Scopus
Can also use Google Scholar
Manually for BCC as may be quicker and easier at the point of inclusion at full text
Generally not double screened
Who:
Either information professional or researcher
When:
Towards the end of the project once the included studies are known
Issues:
-Time consuming as need information from reviewer/researcher about included studies
-When you find more references to include, do you do citation chasing on those?
-Web of Science and Scopus often have different results so may need to search both
-FCC can give more recent references
-Can be making a decision using little information eg if manually BCC then you will only have the title to go on
-Can use publish or perish to download google scholar references
2. Hand searching
What:
Scanning through either journal contents pages or specific conference proceedings, generally done electronically.
This is generally done because some journal titles are not indexed in the databases which have been searched, those journal titles which have repeatedly been shown to have relevant references might have some additional relevant references which your database search did not pick up or they have supplements or special issues which don’t make it onto the databases
How:
Generally done online
Who:
Researchers as they know more about the project
When:
Towards the end of the project
Issues:
-How far back do you go
-Lack of understanding by the researchers what hand searching means (some think it’s just looking at ref lists)
-Should it be carried out by two researchers
3. Internet searching
What:
Searching using an internet search engine
Who:
Can be carried out by either researcher or information professional or done between them
How:
Using any search engine including google/google scholar, do you do both as slightly different results
When:
At the very beginning of a project as a preliminary search, unstructured searching to get an idea of the literature
Further on in the project as part of the search methods
Issues:
-Does it include website searching and social media searching
-Do you search incognito or use a search engine which does track your search history
(eg duckduckgo)
4. Trial registries
What:
Searching trial registries to find out whether any trials have been undertaken but the results have not
There are also some commercial databases of clinical trials:
Cortellis
Adis insight trials
Trialtrove
Could also browse through company newsletters
When:
Generally when the database search is carried out
Issues
-Free trial registries not easy to search
-Why wasn’t the trial published
5. Organisations and repositories
What:
Searching within specific organisations websites or organisations online repositories
Who:
Can be either/both research and information professional
How:
Organisation list generally comes from topic experts but can be found by other means eg internet searching
Many lists exist of potentially relevant organisations eg ??
Global directory of academic open access repositories
When:
Can be carried out at any point during the project
Issues: ??
6. Contacting authors or organisations
What:
Usually to find out further information about a study you’ve already found
Can be to find out about other studies or references
The list can evolve as new authors or organisations can be suggested
Sometimes to obtain the full text of an article
Who:
If for further information about a study then usually the researcher but if to find out about more studies or references then can be either the researcher or information professional
How:
Names from topic experts on the project
Email them directly
Emailing out on a topic specific listserv or discussion group
Call out using twitter
Can use an author filter in Web of Science
Pubreminer
Issues: ??
Other supplementary search methods we didn’t discuss:
Pearl growing
Spear fishing
Snowballing
Harvesting
Linking out eg using researchgate
Some interesting comments
Supplementary searching is a different way of searching, it can be creative but also time-consuming due to the nature of how it is undertaken.
Might be a good idea to tailor your supplementary search methods to the project/question you are trying to find evidence for eg if you are doing a systematic review of qualitative evidence then searching a clinical trials registry will not find anything relevant.
Difficult to know when to stop with any searching but perhaps the diminishing returns idea works ie when you are no longer finding anything new
Grey literature is not a search method, it’s a type of literature. You use several different search methods to find all the evidence, including grey literature
Would a search summary table help us?
Why call it supplementary searching as it is part of the search methods
Open access publishing could create more places to search eg does more online repositories mean you have to search in more places
If using databases eg Web of Science then access can be an issue
Is searching in non-English databases a supplementary search method?
When citation chasing, which list of references should you use?
Is it necessary to report every method/resource, even if you get no returns?
As word counts in journals is a big issue then please use the appendices to record database and supplementary search methods in more detail as it is useful and important to other people who carry out searching
Thanks to all those who came along and helped with this, we hope we can collaborate more.
Morwenna, Alison and Amanda
Evidence Synthesis Team 
2 Pingback