Morwenna Rogers, Alison Bethel and Amanda Wanner ran a continuing education course at the European Health Information and Libraries (EAHIL) Conference.

The subject of the course?
Supplementary Search Methods: who, why, what, how and when 

picture 1

As a group we decided to publish here what we discussed so thanks go to everyone who took part including: Ralph de Vries, Eva Karin and Chantal Haan

Supplementary searching is carried out as part of a systematic review but the methods are often not reported in a way to be transparent and therefore replicable which are two of the main pillars of systematic reviews.

We started off discussing why you might carry out supplementary searching then looked at different supplementary search methods and what they were, how you might do it, who would do it and when within the project.

Why might we carry out supplementary searching

Not all journals are indexed on the databases you are searching or have access to, some references may not be indexed accurately so your search doesn’t pick them up. Perhaps if you don’t find enough studies from database searching, or, it says it in the protocol. There is evidence in the literature that supplementary search methods do pick up additional studies. 

1. Forward and backwards citation chasing/searching (BCC and FCC)

FCC – when you use a list of references to search and find out who else has cited them, often the systematic review included references
BCC – when you look at the end of a reference to find out if they have cited any further relevant references

Can use databases including Web of Science and Scopus
Can also use Google Scholar
Manually for BCC as may be quicker and easier at the point of inclusion at full text
Generally not double screened

Either information professional or researcher

Towards the end of the project once the included studies are known

-Time consuming as need information from reviewer/researcher about included studies
-When you find more references to include, do you do citation chasing on those?
-Web of Science and Scopus often have different results so may need to search both
-FCC can give more recent references
-Can be making a decision using little information eg if manually BCC then you will only    have the title to go on
-Can use publish or perish to download google scholar references

2. Hand searching


Scanning through either journal contents pages or specific conference proceedings, generally done electronically.
This is generally done because some journal titles are not indexed in the databases which have been searched,  those journal titles which have repeatedly been shown to have relevant references might have some additional relevant references which your database search did not pick up or they have supplements or special issues which don’t make it onto the databases

Generally done online

Researchers as they know more about the project

Towards the end of the project

-How far back do you go
-Lack of understanding by the researchers what hand searching means (some think it’s        just looking at ref lists)
-Should it be carried out by two researchers

3. Internet searching


Searching using an internet search engine

Can be carried out by either researcher or information professional or done between them

Using any search engine including google/google scholar, do you do both as slightly different results

At the very beginning of a project as a preliminary search, unstructured searching to get an idea of the literature
Further on in the project as part of the search methods

-Does it include website searching and social media searching
-Do you search incognito or use a search engine which does track your search history
(eg duckduckgo)

4. Trial registries


Searching trial registries to find out whether any trials have been undertaken but the results have not


There are also some commercial databases of clinical trials:
Adis insight trials

Could also browse through company newsletters

Generally when the database search is carried out

-Free trial registries not easy to search
-Why wasn’t the trial published

5. Organisations and repositories

Searching within specific organisations websites or organisations online repositories

Can be either/both research and information professional

Organisation list generally comes from topic experts but can be found by other means eg internet searching
Many lists exist of potentially relevant organisations eg ??
Global directory of academic open access repositories

Can be carried out at any point during the project

Issues:  ??

6. Contacting authors or organisations

Usually to find out further information about a study you’ve already found
Can be to find out about other studies or references
The list can evolve as new authors or organisations can be suggested
Sometimes to obtain the full text of an article

If for further information about a study then usually the researcher but if to find out about more studies or references then can be either the researcher or information professional

Names from topic experts on the project
Email them directly
Emailing out on a topic specific listserv or discussion group
Call out using twitter
Can use an author filter in Web of Science

Issues:  ??

Other supplementary search methods we didn’t discuss:

Pearl growing
Spear fishing
Linking out eg using researchgate

Some interesting comments

Supplementary searching is a different way of searching, it can be creative but also time-consuming due to the nature of how it is undertaken.

Might be a good idea to tailor your supplementary search methods to the project/question you are trying to find evidence for eg if you are doing a systematic review of qualitative evidence then searching a clinical trials registry will not find anything relevant.

Difficult to know when to stop with any searching but perhaps the diminishing returns idea works ie when you are no longer finding anything new

Grey literature is not a search method, it’s a type of literature. You use several different search methods to find all the evidence, including grey literature

Would a search summary table help us?

Why call it supplementary searching as it is part of the search methods

Open access publishing could create more places to search eg does more online repositories mean you have to search in more places

If using databases eg Web of Science then access can be an issue

Is searching in non-English databases a supplementary search method?

When citation chasing, which list of references should you use?

Is it necessary to report every method/resource, even if you get no returns?

As word counts in journals is a big issue then please use the appendices to record database and supplementary search methods in more detail as it is useful and important to other people who carry out searching

Thanks to all those who came along and helped with this, we hope we can collaborate more.

Morwenna, Alison and Amanda