What can systematic review methods offer business and economics research? Mark Clowes (ScHARR)
This was the second event I have attended recently about systematic reviews. Again this was delivered by someone who had been a health librarian showing that this is much more prevalent in the health field than in business.
Systematic reviews are considered the top of the evidence triangle though some contest this. The Cochrane definition is explicit and states that the aim is to collect all evidence that fits pre-specfied elgibility criteria to answer a specific question; so SRs that don’t include all articles or refine the criteria after the initial search are not true SRs. You should expect to see a fully reported search strategy across multiple databases; something else often lacking in so-called systematic reviews.
A gold standard systematic review is expensive in resources. You can negotiate with the researcher as something that is more like a systematic approach may be appropriate. A full SR can take 18-24 months. You might end up with 10,000 FT articles to screen by hand and the best screeners can generally only do about 500 a day. At ScHARR they have teams of 6+ working on one review; in business there might be one or two at best.
The next bit of the talk was about how all of this can be applied to business. Fisch and Block (2018) [https://link.springer.com/article/10.1007/s11301-018-0142-x] found considerable variance in the understanding of a literature review. This article is particularly noteworthy as they are editors for the journal so writing from experience. Also Arnaud Vaganay writing on the LSE blog [https://blogs.lse.ac.uk/impactofsocialsciences/2018/06/19/to-save-the-research-literature-lets-make-literature-reviews-reproducible/] argues that few management literature reviews include the search strategy so they can be reproduced. Often in business researchers do a broad search and then allow themes to emerge, the opposite way round to in health. There are few recent studies on the use of SRs in business.
As librarians we can be involved at several stages of systematic reviews. Some librarians have become fully fledged systematic reviewers but there are ways to go beyond just doing searches for people. We can be involved in the scoping process, advise on methodology for example but we should ensure we get authorship of the paper.
After the keynote there were 6 members sharing papers and a world cafe discussion. Here are summaries of the members papers:
Transforming from a teaching-oriented culture to a research culture
Merger came with expectation to do more research. Gwen did her student dissertation on this topic. Looked at profile of prolific researchers. Realised academic organisations were ambidextrous. Individuals exploiting and exploring at same time. In business would do one or other. Kangaroo and Snell, 3 forms of intellectual capital. Looked at research culture. Booth et al 2016, p21. For her dissertation she took a systematic approach to the literature review. She did a broad search on research culture and university/HE/culture. Screened down to 50 articles then reading full text reduced down to 14. As there were only 14 she didn’t refine further but put all into Nvivo for coding. Key learning: essential to understand why it’s important to be research active, need socialisation, external training important, funding for staff to do PhDs, protect research time over teaching load.
Integrating IL skills for first year management students: reflections
Bristol introduced a new unit (aka module) in management focussed on skills development. Main assessment follows systematic review principles including an appendix detailing search strategy, students had to include 10 academic sources. 12 2-hour workshops in week 3. Students had to do own search in advance. In workshop Business Source Complete gave very different results to their own searching. Students don’t know difference between sources they find. In the session Sarah had time to focus on how database works, using google scholar or discovery engine makes process look easy but students don’t have tools to correct if search goes wrong. CABS list double edged sword as over reliant on this and not developing critical thinking skills. Saw 120 students out of cohort of 180. Involved from start via membership of T&L committee.
Summative assessment of first year undergraduate library skills
Mandatory module 650 students; critical thinking and decision making. Lots of opposition at programme board to a skills-based module. Eventually passed. Critical Thinking for Business. Ronan was tasked with developing a multi-part intervention to introduce students to CT . Week 4 students had to work through online resource. Week 5 six plenary sessions, had to sign in to Moodle to prove attendance. Used Kahoot, Padlet and Mentimeter in session. Week 5 data exercise had to use Passport and Marketline. Week 6 20 question quiz worth 15% of module mark, linear, correct answers not released until quiz submitted. Ronan found the process quite stressful, especially having to devise questions for a summative assessment. A learning technologist started the next week and Ronan has met them already and they will provide assistance for next year.
Very international conference, only 2 from UK, mostly business librarians – not just HE, also academics. Talks included one on researcher support and OA: academics obsessed with getting published in top journals rather than open access, predatory journals more prevalent in management; another on information literacy and digital badges: Wendy Pothier, IL as a workplace skill; new business models, no IL sessions, employed data scientists instead of librarians. Conference every three years
Phil Reed and John Hynes
Helping non-specialists to teach specialist workshops
Manchester moved to functional teams. Non-subject specialists now deliver training. Fine with most databases but business ones are quite specialised. Challenge how to deliver specialists workshops by these non-specialists. Eg Datastream! It’s difficult. Limited support, big folder. Librarians don’t have the subject knowledge. Approach was to get a few steps ahead of the students by training the trainer. Staff were set challenges to do ahead of the session, there were discussed in the session, more content was delivered, more homework, more discussion. Used medium.com to write lesson content as teaching it. It did work, increased integration. Had matrix of trainers and how they rated themselves on each session, aim to change colour for at least one session each semester
Reviewing the situation
Reviewing subs. Tend to look at cost per use but there are other things to consider. What resources are used in courses, requires resource knowledge and reading course pages. Also looked at research outputs. Used Endnote. Then file checked for names of each resources. Can see what the databases are actually being used for. Produced impact map by looking at each paper that came from outputs to see citations. Price and usage shouldn’t be the only thing to consider. Can be used to identify gaps.